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ABSTRACT 



This project for large-group, problem-based learning at 
Kansas State University College of Veterinary Medicine developed 47 
case-based videotapes that are used to model clinical conditions and also 
involved veterinary practitioners to formulate true practice cases into 
student learning opportunities. Problem-oriented, computer-assisted 
diagnostic programs were also modified to help veterinary students increase 
their skills in differential diagnosis. The study was also successful in 
evaluating the neurological problem-space of third-year veterinary students, 
both in initial groups and subsequently as individuals, distinguishing 
skilled from novice diagnostic reasoning. The study found that problem-based 
learning fostered significant advantages and increased student confidence and 
in-depth learning. Disadvantages included reduced breadth of learning, 
considerable time requirements, and significant ambiguity. Twelve of the 
tapes are being used at two other veterinary schools. Sections of the report 
are devoted to an extensive list of the tapes produced and in production; a 
bibliography of materials produced by the project, in addition to the project 
overview, purpose, background and origins, and detailed evaluations and 
findings. Appendices include "Evaluator Analysis of Interviews" (John 
Pickrell, Susan Santos, David Balk) undertaken with 35 veterinary medicine 
students, predominantly in their junior and senior years. (Contains 142 
references.) (SW) 
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ENHANCING LARGE-GROUP PROBLEM-BASED LEARNING IN VETERINARY 
^ MEDICAL EDUCATION 

Abstract: In this FIPSE project, we extended large group, problem-based learning to validate and 
begin dissemination of a video-enhanced, practitioner-based, team-oriented approach to case-based 
veterinary medical education. Forty-seven case-based videotapes were produced and are used to 
model clinical conditions at KSU College of Veterinary Medicine, increasing student involvement and 
the quality of their learning. Learning appears greater if oral examination questions are ordered from 
simple to complex. The neurological problem-space of 3rd year veterinary students was evaluated 
most successfully in initial groups and subsequently as individuals, distinguishing skilled from novice 
diagnostic reasoning. Our evaluation team and investigator triangulated findings reveal that PBL 
fosters significant advantages, increased student confidence and in-depth learning. However, PBL s 
significant disadvantages, reduced breadth of learning, considerable time requirements, and significant 
ambiguity while learning are regarded as barriers to introducing PBL curriculum wide in many higher 
educational institutions. 

Project Director: 

John A. Pickrell 

Comparative Toxicology Laboratories 
Department of Clinical Sciences 
College of Veterinary Medicine 
Kansas State University 
Manhattan, KS 66506-5606 
Telephone (913)-532-4331 

(Fax: 913-532-4481; E-mail: Pickrell@vet.ksu.edu) 

PRODUCTS: 

Cased-Based Videotapes in Veterinary Medicine and Agriculture: 

College of Veterinary Medicine, Manhattan, Kansas, Kansas State University, Supported by 
Supported by FIPSE, Fund for Improvment of Post Secondary Education, US Department of 

‘Supported by both FIPSE and HECG (USDA-Higher Education Challenge Grant) Projects (those 
without asterick * are supported by FIPSE alone). 

A. Completed ^available for purchase at $ 50 /yide otape: order from project director) 

1992 Fortney W and Pickrell JA. Pruritic dermatitis and chronic cough in a dog (Case study). 
1992 Lorenz, MD, Pickrell JA, Fortney W. Dermatology grand rounds, Willie Perkins (iatrogenic 
steroids). Both tapes are discussion of a owner-induced case of 7 years duration with 

dermatology and pneumonic signs. - 

1992* Pickrell, JA. Ammonia and particles in a swine confinement building. Demonstration ot 
measurements and evaluation of indoor air pollution in a swine confinement operation. 
1992* Pickrell, JA, SR Kerr and S Graff: Dyspneic cows on a weed pasture (pulmonary 

adenomatosis). 

1992* Pickrell, JA and Graff S. Dyspnea in cows on a weed pasture. Both tapes discuss cows in 
a weed pasture are dying with pneumonic symptoms. Owners suspect endotoxin involvement. 
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1992 

1992 

1992 

1992 

1992 

1992* 

1992 
1992* 

1993 
1993 
1993* 
1993 
1993 
1993 

1993 

1993* 

1994* 

1994* 

1994 



PickreU, JA. The clinical toxicology oral examination. Our method of oral examination is 

shown for the junior class in 1991. . 

Layton, CE and Pickrell JA. Annie (removal of a canine mastocytoma). Surgical technique 

for removal of a mastocytoma are shown as a surgical problem. 

Oehme FW Heller T and Pickrell JA. What’s wrong with my horses, doc? 

PickreU JA, Heller T, Huston L, Gigstead J, and Oehme FW. My horses are lame, what 
do I do ’(selenium toxicity)? Both tapes show client explanation, equine symptomatology, 
poisonous plant identification and short and long term treatments are discussed in the case- 

CowanL, Pickrell JA and Kerr SR. An ataxic dog (ethylene glycol). Case-based discussion 
of an ethylene glycol intoxication. Diagnosis and management are shown 
Pickrell JA Kerr SR and Epp EL. The case of the wobbly cows (organophosphate 
insecticide). A puzzling case of CoRal intoxication is presented by the referring veterinarian 

Layton, 3 CE and Pickrell JA. Alice, a case of an abdominal mass (removal of a surgical 
sponge). A valuable opportunity to consider sequelae to in implanted surgical sponge is used 

in the case-based format. . r 

Beeman K Pickrell JA, Kerr SR, Kennedy G, and Cimon K. The mysterious case o 

diarrhea from calves from Dodge City. A presumed arsenic herbicide intoxication is shown 

in the case-based forat. . ^ 

PickreU, JA and Kerr SR. Student-centered large group problem-based learning. Our 

method of teaching toxicology is presented in video with accompanying discussion 
Presnell, T, PickreU JA, Kerr SR, and Fineman L,: A big white weak dog (warfarin 

toxicity) Clinical features of toxicity, treatment and client education are featured. 

Gaughan, E, Kerr SR and PickreU JA. "Show me how you think" , demonstrations of clinical 
decision making. The decision process in equine colics is featured. 

Gaughan, K, PickreU JA and Kerr SR. What’s shakin? (Pyrethroid [Defend] neurotoxicity 
in a cat) ' Clinical features, environmental aspects, and managements are featured. 

Gaughan K PickreU JA, Stark N, Kerr SR, and Owen T. A cat of a different color (tylenol 
toxicity) ’ Diagnosis and management of tylenol toxicity are featured. „ . 

Gaughan, K, PickreU JA and Kerr SR. What’s wrong with this cat: Hemobartonella in a cat. 
Diagnosis and management of hemoparasitosis in a cat are featured. 

Baier J PickreU JA and Kerr SR. Leapin Lizards, what happened to Morton: Poor health 
in a monitor lizard. Poor health in a monitor lizard are discussed in reaching a tenative 

Nkldtfeld' J, PickreU JA and Kerr SR. If a Bird in the Hand is Worth Two in the Bush. 
Unexplained deaths in quail used for game are explored. All 3 tapes are shot on the quail and 

nheasant farm at Solomon, Kansas. , ■ 

Niedtfeld, J, PickreU JA and Kerr SR. Pheasant blindness. Pheasant blindness presumably 

from firewe’ed (kochia) intoxication provided as a significant portion of their diet 
Niedtfeld J PickreU JA and Kerr SR. This is a game farm; Raising pheasants, chukars and 
quail for’ sport. Pheasants raised as game birds are discussed as a vector to immerse 

nrosDective exotic practitioners as students. , 

Morrisev J Baier J, PickreU JA, and Kerr SR. Prairie Dog Days: Nervous disorders in 
prairie dogs'- Nervous signs of unknown origin are considered in an effort at proposed 

management. 
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1994* 

1994* 

1994* 

1994 

1994 

1995 



1995 

1995* 

1995* 

1995 

1995 

1995* 

1995* 

1995* 

1995* 



McVey S, Pickrell JA and Kerr SR. Salmonellosis in beagle puppies. Experimental 
salmonellosis in beagle puppies is discussed to introduce the topic to basic bacteriology 
classes, and as a review in clinical rotation. 

Kerr, SR, Anderson NA, Pickrell JA, et al. Beef Jerky, Transitory ataxia in a bull. CNS 
convulsions in a beef bull with encephalomalacia are featured. 

Kerr SR, Anderson NA, Gaughan E, Pickrell JA, Leith D, and Schoneweis D. Tracking in 
Veterinary Medical Education. What’s your opinion? Tracking’s value and the effect of 
collaborative (problem-based; case-based) learning are discussed by KSU instructors. 

Kerr SR, Fortney W, Lorenz MD, and Pickrell JA. The problem with BJ: Cardiac 
decompensation in a dog. Cardiac decompensation in this dog features clinical features and 

response to management. . . , , ... • 

Kerr SR, Baier J, Pickrell JA. Something Fowl is A foot: Surgical repair of bumblefoot in 

a raptor. The clinical features of bumblefoot are shown and its effect on the raptor are 
delineated. Surgical repair of bumblefoot and its response is shown. 

Kerr, SR, Lewis DS and Pickrell JA. Cerebellar hypoplasia in young puppies (11 tapes - 
8 completed) These tapes are conceived to allow students to perform a step-by-step 
neurologic examination, to compare the findings to those of that examination, to formulate a 
diagnosis and differential diagnoses and to make management recommendations. 

• You are the neurologist, #1 Sadie 

• You are the neurologist, ti2 Lavendar 

• You are the neurologist, #3 Major 

• You are the neurologist, M Rocky 

• You are the neurologist, ti5 Zeus 

• You are the neurologist, #6 Argus 

• You are the neurologist, til Toby 

• You are the neurologist, tiS Max 

Layton CE Kerr SR, Pickrell JA, Foxy, A case of persistent vomiting. Hepatoportal shunt. 
Morrissey J, Carpenter J, Kerr SR, Pickrell JA, Immobilization techniques and equipment 
for zoo and wild animals. Optimum clinical restraint of a variety of wild animal species is 
shown for interns, residents and exotic specialists. 

Kerr SR, Morrissey J, J Carpenter, JA Pickrell et al. , Hold your Tigers, giraffes, zebras and 

bisons...' Restraint in zoo exotic species for students and practitioners. 

Gaughan E, Kerr SR, Pickrell JA, Surgical repair of the third degree perineal laceration in 

the mare (June). ... • . 

Gaughan E, Kerr SR, Pickrell JA, The marred mare (June). Perineal laceration and repair 

are discussed. The conditions surrounding the problem and the followup care are delineated^ 
Kerr SR Pickrell JA etal. Big Problems— central nervous system disturbance in a hereford 
bull,’ possible lightning stroke. The differential diagnoses of central nervous system 

disturbances in a Hereford bull are related to a brain tumor (July). 

' Kerr, SR, Pickrell JA, et al., Educational impact of videotaped clinical cases. Senior 

rotational analyses of videotaped cases. 

• Kerr SR, Pickrell JA, et al. , Blind bull. The differential of blindness in a bull are discussed 
and related to this bulls clinical signs. 

■ Kerr SR, Pickrell JA, et al., This is a swine farm. Clinical and animal science aspects ot 
different types of swine farms are visually illustrated. 



1995 Layton CE, Kerr SR, PickreU JA, et al. , A case of a continuous heart murmur. Differential 
for a persisting heart murmur are discussed and recommendations made. 

1995 Carpenter J, Kerr SR, PickreU JA, et al. , Inclusion body disease in a python. Includion body 
disease and associated differential diagnoses and management are discussed for this snake. 



1994 

1994 

1994* 

1994 



B. Editing . Q 

Kerr, SR, DS Lewis and JA PickreU. Cerebellar hypoplasia in young puppies (11 tapes-8 

completed/3 in editing). These tapes are conceived to allow students to perform a step-by- 
step neurologic examination and to compare the findings to those of that examination. 
Fortney, W, JA PickreU and SR Kerr, Demodecosis in a dog. Clinical features outlining 



the condition are shown. 

Schoneweis, D, G Kennedy, SR Kerr, and JA PickreU. Garden-hose gut in swine. 
Prominent features of necropsy are compared to those of normal swine intestine. 

Kraft, SR, JA PickreU, SR Kerr, et al. Digitizing radiologic images are used to demonstrate 

radiologic examinations. 



C. Filming . ... 

1994* Stokka J JA PickreU and SR Kerr. Respiratory difficulties in young feedlot cattle. 

Respiratory difficulties of young cattle entering feedlots and on full feed are described. 
1994* Gaines, J, JA PickreU, FW Oehme, and SR Kerr. Mastitis in dairy cattle. The atmosphere 
of a working mid-sized dairy farm is given as a setting for mastitis in that dairy herd. 

1994* Brandt, R, JA PickreU and SR Kerr. Grain overload-respiratory difficulties in feedlot steers. 
Grain overload causing the death of several calves on feed is complicated by significant 
accompanying respiratory disease. 



COMMUNICATIONS: 



PubUshed Abstracts: • . . 

1991 PickreU, JA: Enhancing large-group problem-based learning in veterinary medical education. 

Annual Program Directors Meeting Abstracts, Department of Education, Fund for 
Improvement of Post-Secondary Education, 1991 Program Book (Project Descriptions and 
Staff Essays, p 84, Washington, D.C. 

1992 PickreU, JA, FW Oehme and VL Clegg: Problem-based learning - a tool to enhance 

toxicology education. Toxicologist 12 (1): 351 . 

1992 PickreU, JA: Enhancing large-group problem-based learning in veterinary medical education. 
Annual Program Directors Meeting Abstracts, Department of Education, Fund for 
Improvement of Post-Secondary Education, 1992 Program Book (Project Descriptions and 
Staff Essays, p 85, Washington, D.C.) 

1993 Kerr, SR, JA PickreU, FW Oehme, CL Layton, D Balk, V Primer, VL Clegg, JR Staver, 

J Galland, WD Fortney, and MD Lorenz: Case-based learning-a tool for involvement, 

empowerment and evaluation in toxicology. Toxicologist 13(1): 274 

1993 PickreU, JA: Enhancing large-group problem-based learning in veterinary medical education. 
Annual Program Directors Meeting Abstracts, Department of Education, Fund for 
Improvement of Post-Secondary Education, 1993 Program Book (Project Descriptions and 

Staff Essays, Washington, D.C.) 



1994 

1994 



1994 



PickreU JA SR Kerr, FW Oehme, CL Layton, D Balk, V Primer, K McKay, VL Clegg, 
F Papa, JR Staver, WD Fortney, and MD Lorenz: Case-based learning for validation and 
professional growth in toxicology. Toxicologist 14(1): 92 

PickreU JA, SR Kerr, FW Oehme, CE Layton, JR Gillespie, DE Balk, V Primer, K McKay, 

V Clegg JR Staver, WD Fortney, MD Lorenz, S Gearhart, J Powell: Case-based 

Learning-A tool for validation and professional growth in veterinary medicine. Proceedings 
for the XIV PanVet Conference, October 9-14, Acapulco, GRO, MX 

PickreU, JA, SR Kerr, FW Oehme, CE Layton, JR Gillespie, DE Balk, V Primer, K McKay, 

V Clegg JR Staver, WD Fortney, MD Lorenz, S Gearhart, J Powell: Case-based learning 
- A tool for validation and professional growth in veterinary medicine. ACVS-AAVC 
Education Section. Published in the large animal section of Proceedings of ACVS, October 

16-19. 



Articles-Chapters-Limited Circulation Reports: , . 

1991 PickreU, JA: Problem based learning - a tool to enhance post-collegiate education in 

Toxicology. Central Society of Toxicology Bulletin. Central States Chapter Society of 

Toxicology Newletter 6(2): 11-14 (invited paper). 

1992 PickreU, JA: Progress Report: Problem-based learning to enhance veterinary me ica 

education. FIPSE, Department of Education, Washington, DC, April 15. 

1993 PickreU, JA: Progress Report: Problem-based learning to enhance veterinary medical 

education. FIPSE, Department of Education, Washington, DC, April 15. 

Invited Papers-Presentations: 



1992 



1992 



1993 



1993 



1993 



1993 



PickreU, JA, FW Oehme and VL Clegg. Problem-based learning - a tool to enhance 
toxicology education. 31st Society of Toxicology Meeting, Seattle, Washinton February 
PickreU, JA, FW Oehme, CE Layton, VL Clegg, JR Staver, PL Daisey, WD Fortney, MG 
Shroyer and MD Lorenz. Problem-Based Learning-A Tool for Involvement, Empowerment 
and Evaluation in Veterinary Medical Education. Twelfth Veterinary Medical Education 
Symposium, Iowa State University, Ames, Iowa, June 13-15 

Kerr SR JA PickreU, FW Oehme, CL Layton, D Balk, V Primer, VL Clegg, JR Staver, 
J Ga’lland WD Fortney and MD Lorenz. Case-based learning-a tool for involvement, 
empowerment and evaluation in toxicology. Presented at 32nd Society of Toxicology 
Meeting, New Orleans, Louisiana 

Kerr SR, JA PickreU, FW Oehme, CE Layton, D Balk, VL Clegg, JR Staver, J Galland, 
WD Fortney and MD Lorenz. Case-based learning-A tool for involvement, empowerment 
and evaluation in toxicology. Central States Chapter, Society of Toxicology, Creighton 
University, Omaha, Nebraska, April 16 

PickreU JA. Enhancing large-group problem-based learning in veterinary medical education. 
Presented at 1993 FIPSE (Fund for Improvement of Post-Secondary Education) Program 
Directors Meeting, Washington, DC, October 8-10 

PickreU, JA: New adventures in group dynamics. Large-group problem-based learning at 
Kansas: Successes and barriers to introduction into the curriculum. Presented at 1993 FIPSE 
(Fund for Improvement of Post-Secondary Education) Program Directors Meeting, 
Washington, DC, October 8-10 




6 



7 



1993 Pickrell, J A (Moderator). New adventures in group dynamics. Collaborative learning in the 
medical sciences and higher education. Presented at 1993 FIPSE (Fund for Improvement of 
Post-Secondary Education) Program Directors Meeting, Washington, DC, October 8-10 

1993 Pickrell, JA (Principal Investigator, Coordinator). Site Visit: Problem-based learning in 
Veterinary Medical Education. United States Department of Education, Fund for 
Improvement of Secondary Education (FIPSE), Dr. Sandra Newkirk, Site Visitor, FIPSE, 

Washington, DC 

1994 Pickrell, JA, SR Kerr, FW Oehme, CL Layton, D Balk, V Primer, K McKay, VL Clegg, 

F Papa, JR Staver, WD Fortney, and MD Lorenz. Case-based learning for validation and 
professional growth in toxicology. Presented at Society of Toxicology, Dallas, Texas, March 

1994 Pickrell, JA, SR Kerr, FW Oehme, CE Layton, JR Gillespie, DE Balk, V Primer, K McKay , 

V Clegg, JR Staver, WD Fortney, MD Lorenz, S Gearhart, and J Powell. Case-based 
Learning— A tool for validation and professional growth in veterinary medicine. Proceedings 
for the XIV PanVet Conference, October 9-14 

1994 Pickrell, JA, SR Kerr, FW Oehme, CE Layton, JR Gillespie, DE Balk, V Primer, K McKay, 

V Clegg, JR Staver, WD Fortney, MD Lorenz, S Gearhart, and J Powell. Case-based 
Learning — videotaped illustrations. Proceedings for the XIV PanVet Conference, October 9- 
14 

1994 Pickrell, JA, SR Kerr, FW Oehme, CE Layton, JR Gillespie, DE Balk, V Primer, K McKay, 

V Clegg, JR Staver, WD Fortney, MD Lorenz, S Gearhart, and J Powell. Case-based 
learning— A tool for validation and professional growth in veterinary medicine. ACVS-AAVC 
Education Section. Large animal section of Proceedings of ACVS, October 16-19 

1995 Pickrell, JA, SR Kerr, FW Oehme, CE Layton, JR Gillespie, DE Balk, V Primer, K McKay, 

V Clegg, JR Staver, WD Fortney, MD Lorenz, S Gearhart, and J Powell. Case-based 
learning— A tool for validation and professional growth in veterinary medicine. Problem- 
Based Learning in Veterinary Medicine, San Diego, CA, March 4-7. 

Preliminary In-House and Regional Presentations 

1989 KSU/College of Veterinary Medicine "First Friday Lunches" Problem-Based Learning 
Experiences in KSU the Veterinary College (JA Pickrell, B Fenwick, W Cash, DTroyer), 
Manhattan, KS, December (attendance _> 40 faculty); 

1990 KCRCHE Teaching Effectiveness Conference, Rockhurst College, Kansas City, KS, 

February) LG-PBL: The early experiences in Veterinary Medicine (attendance >.150 

faculty)(JA Pickrell, W Cash, D Troyer). 

1990 Graduate course EDCI 843, Principles of College Teaching, LG-PBL in the veterinary 
toxicology classroom, Kansas State University, Manhattan, KS; April (JA Pickrell, attendance 

_> 10 potential faculty). . , 

1990 Master Teachers Conference, Case-based learning in the professional curricula, St. Mary s 
College, Leavenworth, KS; June (attendance _> 25 faculty)(JA Pickrell). 

1990 KSU/College of Veterinary Medicine "First Friday Lunches", SG-PBL in Veterinary 
Pathology: an experiment, Manhattan, KS, August (attendance >. 40 faculty)(R Oberst), 

1990 KSU/College of Veterinary Medicine "First Friday Lunches" , Cooperative learning: the best 
approach? Manhattan, KS, December (attendance >l 40 faculty)(B Fenwick); 

1991 KSU/College of Veterinary Medicine "First Friday Lunches", LG-PBL in Veterinary Clinical 
Toxicology: the first 2 years, Manhattan, KS, April (projected attendance 40 faculty)(JA 
Pickrell, FW Oehme) 
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ENHANCING LARGE-GROUP PROBLEM-BASED LEARNING IN 
VETERINARY MEDICAL EDUCATION 
Grantee Organization: Kansas State University; Department of Clinical Sciences 

Comparative Toxicology Laboratories; Manhattan, KS 66506-5606 

Project Director: John A. Pickrell 

Telephone (913)-532-4331 [Fax (9 13)-5 32-4481; E-mail Pickrell@vet.ksu.edu] 
EXECUTIVE SUMMARY: Project Overview: Real-life questions were being posed to junior 
veterinary students in a Toxicology class in the College of Veterinary Medicine at Kansas State 
University (KSU) to initiate discussion of diagnosis and management of "their cases." These 
students are part of a growing number who learn by analyzing practice cases. In this project, 
we extended this learning to validate and begin dissemination of a video-enhanced, practitioner- 
based, team-oriented approach to case-based veterinary medical education. During this project 
we produced 48 case-based videotapes (3 more filming or 6 in production) and a small animal 
and a ruminant neurologic diagnostic matrix. We tested and evaluated the small animal 
diagnostic matrix. We find that the tapes were memorable when used in class, as indicated by 
student comments to instructors and on anonmyous surveys. Twelve of these tapes are being 
used to instruct veterinary classes at veterinary schools at the University of Texas and the 
Atlantic Veterinary College on Prince Edward Island, Canada. We believe the tapes have their 
greatest utility as review tools when senior students model clinical conditions during their senior 
clinical rotations. KBIT estimated student small animal neurologic knowledge bases in both 
groups and individuals. Group interactions and added time for integration allowed students to 
bolster their perceived weaknesses more effectively than did individual study. 

Purpose: We enhanced large-group problem-based learning (LG-PBL) to improve student 

learning by 1) developing cases into case-based videotapes and by involving veterinary 
practitioners to formulate true practice cases into student learning opportunities and by 2) 
Modifying problem-oriented, computer-assisted diagnostic programs so that veterinary students 
could enhance their skills in differential diagnoses. 

Background and Origins: Ideas for this project began to evolve better methods of teaching and 
learning for the 21st century, encouraged by the Pew Trust, discussing the possibility that 
learning could most effectively take place in the context in which it would later be used. Our 
faculty team interested in case-based learning was assisted by adaptation of Dr. Frank Papa’s 
(TX College of Osteopathic Medicine) knowledge bit information (KBIT) computer system for 
assessing student knowledge bases for veterinary medical diagnoses. 

In adopting LG-PBL as our teaching strategy, it was our intention to change student 
learning strategies from memorization to learning throughout their professional careers. 
LG-PBL requires that students actively work toward the understanding or resolution of specific 
problems. In working with the problems, "students are expected to draw on previous learning 
and experience, to pose questions concerning new issues, to set personal learning goals, to take 
responsibility for their own learning through independent reading and study, and to teach one 
another with student-to-student discussion" (141). 

Project Descriptions We enhanced LG-PBL by: 

1. Developing the case-based practitioner-assisted videotapes to formulate true practice cases 
into student learning opportunities. 
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2. Modifying existing problem-oriented computer-assisted diagnostic programs (KBIT software) 

to enhance student differential diagnostic skills. 

To accomplish these objectives, we validated 47 case-based videotapes, and canine and 
ruminant KBIT analyses of neurotoxicant space. We disseminated our learning system 
throughout our college and into two other North American Veterinary Colleges. Academicians 
from Mexico, Brazil and other Pan American Veterinary Colleges expressed interest in the 
possibility of translating our case-based scenarios which apply to their unique areas. Visually 
enhanced stories were filmed and produced as case-based videotapes. These tapes were used 
as the framework for advanced learning in junior clinical toxicology and mentoring clinical 
toxicologic medicine during senior rotations. Groups improved student involvement and 
intragroup cooperation. Students began to validate their accomplishments by testing their 
knowledge bases against KBIT following their experience as junior groups and by answering 
randomly drawn "quiz-bowl" types of questions on their senior rotations. Transition to limited- 
temporary expertise was quantifiable and dependent upon previous experience as indicated on 
oral examination. This fall we used both group and individual questions to obtain group 
improvements in treatment, diagnostic reasoning and ability to explain mechanisms, the 
beginning of expertise. One month later, the group experience was associated with slightly 
higher scores on examinations similar to that given 1.5 weeks earlier. In addition, the group 
experience was associated with more treatment completeness and greater depth of explanations 

of mechanisms, suggesting further steps to expertise. . 

Evaluation/Project Results: Students find the case-based videotapes of considerable use in class 
for introduction and modeling to that clinical condition as indicated by unsolicited comments 
following their presentations and comments on surveys. In addition, they find them of 
considerable use for reviewing during their clinical rotations. Many have indicated interest in 
purchasing new tapes in practice, as have practitioners at annual conference and veterinarians 
from Mexico, Brazil and other panAmerican countries (translation needed). They make 
productive suggestions for improvement of tapes and find they present a clear model of the 
signs— almost like a case. Faculty who use these tapes believe that such models go beyond word 
paper descriptions, but fall short of an excellent clinical case, where all the senses may be used 
for diagnostic reasoning. These tapes allow the case to be reviewed until concepts are clear. 

Teaching partners, a partnership between CE Layton and JA Pickrell in which we 
visited each others classes 3 times and interviewed 3 students each, making and soliciting 
formative comments to improve teaching, revealed a bias toward action (hands-on experience), 
increasing use of groups for consultation, appreciation of the instructor support provided by 
these teaching partners, and a preference for individual oral examinations, especially those 

exainations with questions of ordered difficulty. 

During the second project year D Balk’s evaluative team confirmed widely held college 
beliefs and college issues about problem based learning (PBL). Both strengths and potential 
problems of PBL were those reported in the PBL literature. Widespsread beliefs) confirmed 
by interviews were: 1) PBL takes a lot of time; 2) Ambiguity is inherent in PBL; 3) 
Facilitators are the key to student success in small-group PBL; and 4) Learning the problem- 
solving process is a goal of PBL. Issues confirmed dividing faculty, students and 
practitioners were: 1) Combining lectures and PBL would be more effective either separately; 
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2) PBL builds student confidence; 3) PBL increases retention of information; and 4) PBL allows 
in-depth learning at the expense of breadth of coverage. 

Evaluation of the 3rd year Clinical Toxicology students conducted during the 2nd and 
3rd project years indicated partial validation of these objectives (investigator comments in italics) 
l)Faculty provided a structure which will allow students to integrate what they have learned and 
enhance student transition to temporary experts; students resisted this structure; 2) Faculty 
provided exams fostering an instructor-dependent colleague to colleague relationship to facilitate 
learning; 3) Faculty developed tests which reflected student abilities; 4) Students only partially 
changed learning strategies from memorization to lifelong learning; 5) Education was initially 
direct; as demands in other courses increased learning became passive. 

’ Case studies of 3rd year veterinary students conducted during the 3rd and 4th project 
years revealed that: 1) Students are given large amounts of information causing them to fall 
behind; 2) Student’s lives reflect their scheduled hourly tests for the week; 3) Students use their 
spare time to do things that they enjoy; 4) Students consider an A or a B an acceptable grade; 
5) Groups support individual members, serving an important function in the lives of veterinary 
students; 6) The 3rd professional year marks the transition between the first 2 years of basic 
science and the 4th clinical year just preceeding students entry into their practice life; 7) The 
main concern of 3rd year professional students is preparation for the 4th year; 8) Students do 
not believe the national board is an effective method of testing their skills as practicing 
veterinarians; 9) Students showed signs of burnout during their 3rd professional year; 10) 
Students respect faculty who treat them with courtesy (willing to help outside of class, grade 
exams promptly, treat them like professional colleagues) and who have significant practical 
experience (knowledgeable, provide practical information in a way that they can understand). 

During our 4th project year, our team of evaluators and investigators validated the 
following hypotheses: Case-based videotapes Hypotheses 1 and 2) Practitioner-generated 
cases and case-based videotapes increased student involvement as estimated by instructor, 
observer and by videotapes; Hypothesis 3) Case-based learning increased the incidence of 
practitioner quality answers on oral examinations; Hypothesis 7) At least 80% of the 
students did ngt take credit for transition from novice to limited pre-expert; only 10-60% 
did, and that with difficulty; KBIT profiles : Hypotheses 4 and 5) KBIT profiles of the 
neurotoxicant problem space in small animals could distinguish mastery levels of diagnostic 
skills from low levels of diagnostic skills. Students more easily identified areas which needed 
work, and had more difficulty taking credit for their achievements. Hypothesis 6, students with 
high diagnostic efficacy would have a different diagnostic path was not testable by KBIT. 
Summary and Conclusions: Forty-seven case-based videotapes were produced and are used 
to model clinical conditions at KSU College of Veterinary Medicine, increasing student 
involvement and the quality of their learning. Learning appears greater if oral examination 
questions are ordered from simple to complex. The neurological problem-space of 3rd year 
veterinary students was evaluated most successfully in initial groups and subsequently as 
individuals, distinguishing skilled from novice diagnostic reasoning. Our evaluation team and 
investigator triangulated findings reveal that PBL fosters significant advantages, increased student 
confidence and in-depth learning. However, PBL’s significant disadvantages, reduced breadth 
of learning, considerable time requirements, and significant ambiguity while learning are 
regarded as barriers to introducing PBL curriculum wide in many higher educational institutions. 
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ENHANCING LARGE-GROUP PROBLEM-BASED LEARNING 
IN VETERINARY MEDICAL EDUCATION 

Project Overview . . . 

Real-life questions were posed to junior veterinary students in a Toxicology class in the 

College of Veterinary Medicine at Kansas State University (KSU) to initiate discussion of 
diagnosis and management of "their cases.” These students were part of a growing number who 
learn by analyzing practice cases. In this project, we extended this learning to validate and 
begin dissemination of a video-enhanced, practitioner-based, team-oriented approach to case- 
based veterinary medical education. 

At KSU, this form of learning began as part of a self examination and partial solution 
of the ever increasing information required to be learned as part of today’s education to become 
a veterinarian and to practice Veterinary Medicine. College-wide, faculty members intensely 
discussed the possibility that learning could be enhanced by obtaining essential facts in the 
context that they would be encountered in veterinary practice. Our project proposed to extend 
this concept by using practitioner cases to place learning in the context of veterinary practice. 
Using videos of clinical signs to model specific important clinical veterinary diseases. This 
approach was inspired by watching instructors act out clinical signs in a story-telling form and 
subsequent discussions with them. On a routine literature search, JAP discovered an interesting 
use of a computer program where clinical signs and potential diagnoses were cross-compared 
to assess students ability to differentiate and subsequently model clinical conditions. Subsequent 
discussions with Dr. Frank Papa (TX Coll Osteopathic Medicine, Ft Worth, TX; FIPSE project 
2-time awardee) suggested that we could adapt his work to Veterinary Medical situations so that 

enhanced student knowledge bases could be evaluated. 

These teaching interactions at KSU initially involved eight faculty who have used case- 
based learning in a variety of different ways. They report heightened student interest and 
enhanced learning. Our administration strongly supported this revolution. Our numbers had 
grown from one faculty member during the spring of 1989 to nearly fifteen faculty members 
having an interest in problem-based learning (PBL) less than two years later. 

During this project we produced 47 case-based videotapes, a small animal and a ruminant 
neurologic diagnostic matrix. We tested and evaluated the small animal diagnostic matrix. We 
find that the tapes were memorable when used in class, as indicated by student comments to 
instructors and on anonymous surveys. Twelve of these tapes are being used to instruct 
veterinary classes at veterinary schools at the University of Texas and the Atlantic Veterinary 
College on Prince Edward Island, Canada. We believe that the tapes have their greatest utility 
as review tools when senior students model diseases during their clinical rotations. KBIT 
estimated student small animal neurologic knowledge bases in both groups and individuals. 
Group interactions and added time for integration allowed students to bolster their perceived 
weaknesses more effectively than did individual study. 



We concluded that we could enhance student learning with large group problem based learning 
(LG-PBL) by: 
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1. Investing considerable energy in developing the cases and case-based videotapes which 
are at the heart of LG-PBL by involving veterinary practitioners to formulate true 

practice cases into student learning opportunities. 

2 Modifying the existing problem-oriented, computer-assisted diagnostic program KBIT to 
enable students to enhance their skills in differential diagnoses in school and practice. 

Background and Origins . 

"Doctor, what are you going to do for the cows which are down and gasping/ this 

question could have been asked of any large animal veterinarian practicing in almost any english 
speaking country anywhere in the world. Instead, it was posed to junior veterinary students in 
a Toxicology class in order to initiate a discussion of the diagnosis and management of their 
case." These students in the College of Veterinary Medicine at Kansas State University (KSU) 
are part of a growing number who learn via Large-Group Problem-Based Learning as they 

analyze real-life cases. . , . , 

Initially our students had enthusiasm as indicated by their covering most of the assigned 

readings and come to class ready for "their case." A higher percentage of students are actively 
involved’ in learning during the Toxicology classes. People visited our classes to watch PBL "in 
action". In addition, a toxicologist at another university has asked to trade cases. Our students 
recommended the evaluation system used in a case-based course, Clinical Toxicology and 
Surgery because they felt that the exams were effective and accurate evaluations of their ability, 
a learning experience that allowed them to integrate their knowledge to that point. 

In the December 1990, mid-year veterinary medicine juniors were surveyed in order to 
obtain comparative data for faculty who are working to improve the testing procedures for first- 
year students in the College. Juniors were asked for their reactions to the testing procedures 
they had experienced during the previous fall semester. Specifically, they were asked to explain 
"What changes would you suggest to make testing procedures more effective in helping you 
learn and in evaluating what you know." The survey did not mention either case-based or 
problem-based learning. Nevertheless, 20 of the respondents suggested that examinations similar 
to those we were using to assess learning in LG-PBL in Toxicology and Surgery be initiated. 
Their answers revealed why they made these suggestions. They felt that exams were effective 
and accurate evaluations of their ability, that they were a learning experience, and that oral 
exams allowed them to integrate their knowledge to that point. These answers are noteworthy 
for three reasons: 1) students usually express disaffection, not affection for tests; 2) this was 
the only positive solution proposed consistently for the problems students saw with the testing 
procedures; and 3) they chose problem-solving as the best way to test themselves for 
performance in clinical decision making, an activity in which they presumably viewed 
themselves as totally immersed by this stage in their careers. 

LARGE-GROUP PROBLEM-BASED LEARNING (findings, from this project, germane 
to this review are noted in italics: 

In adopting Large Group Problem Based Learning (LG-PBL) as our teaching strategy, 
it was our intention to change student learning strategies from memorization to a style more 
conducive to learning throughout their professional careers, in retrospect, we were at least 
partially successful in accomplishing this goal. LG-PBL requires that students actively work 
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toward the understanding or resolution of specific problems. In working with the problems, 
"students are expected to draw on previous learning and experience, to pose questions 
concerning new issues, to set personal learning goals, to take responsibility for their own 
learning through independent reading and study, and to teach one another through 
student-to-student discussion" (141). In our experiences, students took this responsibility 
reluctantly. Initially, our administration strongly supported this teaching revolution as part of 
a continuing desire to improve veterinary education at KSU. 

We believe that LG-PBL is an improvement over more traditional memory-based 
classroom teaching strategies because LG-PBL allows students to practice solving problems in 
the context in which the knowledge is to be used (2, 13, 58, 65, 100, 101, 141). LG-PBL 
speeds incorporating facts into clinical procedures, and the emergency procedures learned 
become almost automatic (10-12, 26, 33, 58) As Mark Twain said, "A person who has had 
a bull by the tail once has learned 60 or 70 times as much as a person who hasn t (132). 

Problem-based learning engages students by requiring that they actively work toward the 
understanding or resolution of specific problems. In working with the problems "students are 
expected to draw on previous learning and experience, to pose questions concerning new issues, 
to set personal learning goals, to take responsibility for their own learning through independent 
reading and study, and to teach one another through student-to-student discussion" (141). 

Problem-based learning is interactive and student-centered. It focuses attention on 
student learning, not teaching (10-15, 33, 34-38, 52, 56, 58-71, 86-115, 125-135, 140-141). 
The case is a form of storytelling (75, 102, 142) which focuses directly on a practice situation 
and leads to increased student-student-faculty interactions, discussions, cooperation and learning 
(10-12 15 29, 33, 34-36, 44-45, 47, 57- 59, 70, 101, 141). As in most good schools, 
involvement, intensity and’ focus of both students and teachers raises the quality of their 
profession (14, 32, 107). Although students do not enjoy the ambiguity inherent in PBL, 
especially ambiguity under pressure, they learn to enjoy the mystery of the diagnosis (83), and 

through instructor guidance find their future (80). 

Problem-based learning is frequently used to teach higher-order cognitive, affective 
and psychomotor skills (10-12, 34, 44, 56, 60-61); for example, cases provide a close link to 
field experiments, teaching cognitive analysis and evaluation, affective valuing— organization, 
and psychomotor adaptation. The teaching orientation using this method is student-centered. In 
this approach, elements of both student-centered cognitive learning styles and student-centered 
affective learning styles are modeled. Problem-based learning focuses on use of context and 
intuition and facilitates student learning to the point that students can visualize themselves 
effectively making correct diagnoses and validate these diagnoses. While the students haven t 
yet become experts, they are close enough to have confidence that they will eventually achieve 
an acceptable level of diagnostic skill. Experiencing this "pre-expert role" dramatically 
influences professional and post-college education as well as patient healing (13, 51, 55, 57, 63, 
68 91, 100, 106, 119, 126). In our experience, as students mature, they develop the patience 
under pressure to begin to validate their diagnoses during their clinical year. PBL teaches them 

patience with some difficulty. . 

Analysis of a case situation "fixes" the facts by making connections to other similar 

data bases (13, 23, 29, 31, 36, 52, 56, 93, 65, 128, 138) and defining a specific disease 
(definers) The connections and the recall of these facts adds analytical depth, encouraging the 
novice to abandon surface problem-solving strategies in favor of the deeper expert strategies 
(108 117) and leading to increased present and later performance (34). Expert problem solving 
strategies collect, encapsulate, transform, retain, quantitatively explain and refine diagnostic 



hypotheses, going beyond marginal solutions to solve everyday practice problems (74a, 85a, 
126a, 133a). 

Factors conveying the ability to discriminate one disease from other similar ones 
(discriminators) are a function of specific domain knowledge and experience, and may be the 
limiting factor to novice-intermediate clinical performance (22, 69, 95, 96, 97). There are five 
common reasons for diagnostic-therapeutic errors (137): 1. Insufficient data gathered; 2. 

Inadequate hypotheses generated; 3. Inattention-misinterpretation of differentiating cues; 4. 
Delayed or missed diagnoses; 5. Inappropriate-mismanaged-delayed therapy. LG-PBL seeks 
to reverse limitations #1 - #3. Our experience suggests that sign frequency and descriptions, 
monitored by KBIT analyses relate specifically to 2-4. Expertise may relate to ability to analyze 
cases of increasing atypicality and arrive at correct diagnoses. 

In cooperative learning, students take responsibility for their own learning and 
support each other, much as mentors would (68). Teams or groups promote student 
involvement and active, student-centered, cognitive and affective learning. Since most learning 
in real-life is cooperative, students beginning early in elementary school have used this technique 
to maximize their learning. This technique is problem-driven, and problems can come from all 
walks of life, allowing students to model future potential occupations while learning (68). 
Beginning groups function best as pairs or triads. Even accomplished student groups will 
seldom be productive with more than 7-8 members. Many classrooms in which cooperative 
learning is practiced have 18-25 students, but some may range as high as 150-1- students. 

Commonly, a small group of 5-7 students practice problem-based learning with a faculty 
adviser. This teaching approach generates the most student-student interaction. It does expose 
students to the most faculty and their different teaching styles; however, it is extraordinarily 
faculty intensive (teacher /student >_ 1.4;) and is made more so by relying on additional faculty 
experts. Our experience suggests that groups of 5-6 students, mildly challenged, or groups of 
5-6 taking typical case-based examinations work together better than groups of 3-4 3rd year 
veterinary students. 

Problem-based learning can also be practiced in large groups of 30 to 150 with a single 
expert faculty discussion leader. This Large-Group Problem-Based Learning generates more 
student-student interaction than does lecture-discussion. The faculty discussion leaders in LG- 
PBL provide a student-centered affective component to learning. Finally, this application of 
problem-based learning is less faculty-intensive, making it possible to accomplish with the 
present size of almost all veterinary faculties (teacher/student <_ 0.25 at KSU). Our experience 
suggests that LGPBL has inherently higher ambiguity levels and requires more summary than 
does small group problem based learning. This method provides no intragroup working 
important to the success of SGPBL. 

Periodically breaking this large group into several smaller groups is believed to make 
the student-centered, cognitive learning experience more like that of problem-based learning with 
smaller groups by generating relatively higher amounts of student-student interaction. This 
unusual combination of LG-PBL and cooperative learning makes learning more dependent upon 
student involvement, and less dependent upon the individual teacher’s pedagogical technique. 

Even though almost nothing is known about how information can be extracted from 
pictures (71, 79-84), word pictures or actual illustrations appear to magnify the strength of 
problem-based learning (23). We know from experience that information is recalled from 
mental pictures as needed, and that it is never even a small fraction of the total richness and 
context of information that the original picture carried (116). The use of stories presented by 
the practitioner cases on videotape enhanced student learning. 



Increased enjoyment of learning for both students and teachers through the strategies 
of problem-based learning leads to a desire to continue learning beyond college (115, 116, 125, 
131-141), often throughout the professional’s life. 

Project Descriptions 

Our project noted 2 LG-PBL enhancements to significantly improve student learning: 

l Selecting and developing the cases which are central to learning by LG-PBL by 
involving veterinary practitioners in formulating true practice cases into student 
learning opportunities. 

2. Modifying existing problem-oriented, computer-assisted diagnostic programs to allow 
student multiple trials, thereby enhancing formulation of lists of likely diagnoses, the 
differentiation of likely diagnoses, and further description of the professional’s model 
(vision) of the specific disease in school and later in practice. 

X. Visual informational extension of LG-PBL provided greater details defining disease 
models and discriminating such diseases from similar n eurological diseases. We brought 
practitioner-assisted cases to the veterinary medicine classroom as case-based videotapes. 
Context and practice stimulate problem-based learning (2, 58, 65, 141), actual practitioner cases 
improve context. Papa’s (94) demonstrated that the richness of the real case will increase 
involvement in the case and empowerment to suggest diagnostic hypotheses and management 
procedures. KSU’s mixture of small animal practices in the Kansas City, Wichita, and Topeka 
metropolitan areas and their large base of beef production (74) allowed us to draw cases can be 
drawn from veterinarians already practicing problem-oriented small animal medicine (46) and 
from those clinicians practicing the more traditional system/disease-oriented, prevention-based 
beef production medicine at the nation’s center of production medicine. 

We documented 47 cases to make case-based videotapes. Initially, Toxicology Professors 
Oehme and Pickrell identified the most promising toxicology and medicine cases from the more 
than 5,000 telephone consultations and the 1,100-1,500 toxicological analyses relayed annually. 
Cases from our intensive care unit were identified for student learning (SG-PBL, LG-PBL, oral 
examinations, educational issues) community medicine, internal medicine, exotic medicine 
(prairie dogs, snakes, hawks, quail), equine medicine, food animal medicine, bacteriology- 
immunology, radiology-anatomy-internal medicine neurology and soft tissue surgery. We 
solicited practitioners’ most puzzling cases through requests and by word of mouth at the time 
of telephone consultations. This collaboration provided richer cases, and led to closer working 
relationships with our already supportive alumni, fostered by investigator membership in 
committees, committee chairs and officers of the KVMA and in one case the AVMA. Once an 
initial potential case was identified, the most promising cases were be further screened. 
Selection of cases will depend upon teaching value: 1) relevance (frequency), 2) completeness 
(amount of diagnosis-management puzzle present), 3) defining and differentiating features; 4) 
diagnostic accuracy; and 5) management appropriateness and timeliness. 

The case-based videotapes were documented by videotaped personal interviews in the 
veterinarians’ or owners’ practice location. These videotapes provided a wealth of contextual 
detail. Such detail was useful in formulating the case. In addition, the details of clinical signs 
from both discussion and illustration was documented clearly. Students analyzing these cases 
found initial modeling of the case and review of that modeling in their senior rotations to be of 
great value. Direct interviews of practitioner or faculty illustrated these advantages. 




1516 



• Face to face practitioner, faculty and some client contact added immensely to the case, 
exceeding information of the initial telephone interviews because the limited verbal 
communication possible by telephone accounts for only about 15% of the total 
communication possible between two individuals talking face-to-face. 

• Practitioners and faculty set time aside for people for the important discussions of their 

• Clinical details arose from the interaction of two consulting practitioners, just as they do 
in problem-based learning and in life. 

• Practitioners and faculty remembered important details regarding their cases much better 
when they were in their own environment. 

• Interviewing on site will bring the faculty member/interviewer back into the practice site, 
thus improving the interviewer’s appreciation for and understanding of the setting for t e 

Next the coinvestigators and the post-doctoral fellow prepared LG-PBL cases from the 
information and videotaped interviews. Selection of the relevant detail from the case was a 
consensus of the case writer, the practitioner or faculty member, and an expert in that field. 
This knowledge was combined with knowledge relating to comprehension in a n^rative-case- 
story-practice situation (23), and to the pattern and context of that case (87, 116). We will 
provide defining and discriminating clues in a wealth of contextual detail derived from practice. 
This detail made the case hang together more completely, and suggested means for the student 
to organize the essential detail, progressing from storytelling to defining, discriminating and 
finally integrating with conceptual detail (87, 116). We monitored the increase in knowledge 
gained from videotapes relative to that from case analysis at oral examination. 

Following these decisions, the videotapes were edited for classroom use by the staff of 
the Audio-Visual Resources Center (AVRC) who provide comprehensive A V services to the 
College of Veterinary Medicine and to other requestors. When the case is first presented in the 
classroom, the practicing veterinarian or faculty member was invited to participate. 

Under the scrutiny of 90 intensely-focused veterinary students, facts and issues will emerge that 
sharpened and clarified the case and improved its usefulness as a teaching tool. Senior (4th 
year) veterinary students made suggestions for improvements which were implemented. This 
videotape was edited by consultation between the same team (practitioner, interviewer and 
faculty specialist) and revisions incorporated into the case-based videotape. 

2. When nnr veterinary medical students use the s p ecial version of KBIT developed by 
Papa for this project, they selected the most relevant primary complaint, small animal or 
ruminant neurotoxicology. The students designated the relative frequency of that complaint 
(variable) looked at indicators-times of risk (in our example, lead from batteries, rodenticides 
enterotoxemia/food poisoning), as well as modifiers of this primary complaint such as onset 
(when did it happen... minutes to several days ago), duration (it is a continuing condition), and 
intensity (high) Finally, the students looked for correlation with subordinate signs, perhaps 
respiratory signs associated with progressing shock, and secondary association of the liver with 
both shock and dehydration. When the students entered everything they know about the disease- 
symptoms and their relative incidence, they indicate that they have finished This process was 
referred to as "fitting characteristics of their knowledge base to their differential diagnostic 

^ Th e KBIT computer program compared the students diagnostic protocol against the case 
battery indicated the number of cases in that battery that the student would have diagnosed 
correctly and indicated what the incorrect diagnoses had been. A score is reported as percen 
correct for each potential diagnosis. Then the students modified their diagnostic protocol, reran 



the program and immediately note improvements. The KBIT computer program was not able 
to keep running track of changes in their diagnostic protocol and changes in their score. I his 
audit capability was not able to be installed as a new feature of the KBIT software program. 
In our experience, students were much more able to identify diagnostic deficiencies than to 
pinpoint them specifically. This made it difficult for students to take credit for their achievements 
unless they were already quite good diagnosticians. Our best progress was with students who 
had tried this program initially as juniors in groups and then subsequently as individuals. 
Preliminarily, this fall, we have verified the same enthusiasm from senior groups taking it for 
the first time. They inform us that they expect to try it as individuals in the near future. 

LG-PBL students used KBIT in teams to develop clinical decision making strategies. 
Subsequently individual students practiced with this software after hours to sharpen their 
diagnostic skill prior to their next evaluation. To date, such analyses have not yet been made 
a part of our student evaluations, because of the pronounced underestimation of diagnostic 
efficacy in novices and intermediate diagnosticians. F Papa (TX Coll Osteopathic Med) and J A 
Pickrell (KSU) continue to consult about using this as an evaluation of group testing, and against 
case atypicality to evaluate student knowledge bases (36-38, 64, 93-95, 118). 

We adapted Papa’s KBIT software for use in the veterinary medical classrooms to 
improve student diagnostic skills. F Papa and his computer programmer modified the KBIT 
software to our specifications. To date, in spite of investigator discussions, only minimal interest 

has been demonstrated for this in practice situations. 

Dr. Papa joined us in Manhattan to conduct initial introductions; we were able to join him 
in Forth Worth, TX; we consulted frequently by phone. We initiated protocols to investigate 
neurotoxicant problem space in small animals and ruminants, and are continuing this process. 
We most extensively tested evaluations of small animal neurologic space, reflecting the fact that 
75%+ of our students will work in small animal medicine, and the commonality of 

neurotoxicant problems. Results from these processes. 

• Defined group student knowledge bases for the common small animal neurotoxicant 

diagnoses. . . . ,. 

• Allowed students to consult about her/his data base to one more successful (diagnosing 

a higher fraction of cases in that battery). 

• Experientially partially taught students to define a knowledge base. Less proficient 
diagnosticians identified so many potential problems that they had trouble progressing. 
Encouraging them to work in groups seemed to "prime the pump " sufficiently to get them 
started. After reaching a high plateau of diagnostic skill they were able to validate their 

achievements. 

• Partially allowed the student to isolate the facts resulting in the greatest increase in 
efficiency in diagnosing that battery of cases. A goal of these analyses is to isolate 
specific critical facts for students to increase their lifelong knowledge bases. This was 
the most difficult part of this analysis to achieve, and only good to very good 
diagnosticians accomplished this. Only after reaching a high plateau of diagnostic skill 
they were able to give self-credit for their achievements and understand how they got 
there. Future ability to trace paths to expertise will provide valuable comparisons to our 
work with group testing, and F Papa’s work with case atypicality. 

Specific criteria were established to aid this detailed entry. Menus encouraged complete 
entry and the quantitative detail the students possessed at that time (94 , 95). As the KBIT 
software is used, further quantitative detail was added. The KBIT software was a tool to help 
the student understand essential disease discriminators and definers, and to organize such 
understanding into his or her knowledge base. The KBIT program partially facilitated the 
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students’ individual organization of discriminating and defining detail of each disease, increasing 
learning. Near the end of the 3rd year professional Clinical Toxicology course, the students 
were given one week to modify their protocol, and this protocol then was run against the same 
cases for a portion of their course evaluation. This trial was be open-book, time-limited and 
graded as a group of 3-4 students (34, 58, 94, 95). Structuring the disease model by 
understanding symptomatology incidence to differentiate among diseases and confirming the 
suspected disease was monitored using Knowledge Bit Information Transfer (KBIT; 94, 95, 
95a). 

Initial results with LGPBL: In a typical Toxicology class session at KSU, the students 
form into groups of six after a case presentation in order to elaborate the issues of problem 
identification, differential diagnosis, mechanisms, diagnostic confirmation, treatment, and client 
education, just as they will in their future practices. Only rarely are they limited by the issues 
the instructor had intended to cover. The students express different learning styles as evidenced 
by different levels of preparation, free talking, random writing and variable involvement with 
other students and with the faculty. Our students rarely agree, even within their groups. Thus, 
the clash of minds at least accompanies, and may be important to, student learning (24, 33, 47). 
Numerous studies of interactive learning strategies in a variety of settings ranging from 
elementary school to college report increases in student and teacher enjoyment of learning, in 
formal (reflective) thinking, and in learning content (73). 

Students have sufficient enthusiasm so that most cover the assigned readings as 
indicated by their coming to class ready for the case and participating vigorously in the 
preliminary discussion of that case. There was an increase in the student-student and 
student-faculty class interactions, both of which contribute to the formation of mentorships and 
colleagueships that may be important as they enter practice. Students volunteered more 
comments during case analyses. Test scores were better than those from our lecture course of 
two year ago, even though the testing approach was changed from short answer to essay 
responses to case histories of practice situations. Many students had considerably more depth 
of understanding than we required for full credit on the oral examination. This fall, the 
percentage of such students increased with a second, and more than doubled with a third oral 
examination, indicating improved ability to relate to clients in a practice situation. 

Some students reported that problem-based learning required more tune to prepare. 
Students often did appear more anxious prior to their first examination and their oral evaluation, 
probably relating to a perceived inability to discriminate and organize essential clinical detail 
from each toxicant. While they did well on the tests, a survey showed that many did not take 
credit for their accomplishments, and instead attributed it to an external source such as the 
instructor. It is important to alleviate both of these factors by focusing student attention on their 
uniqueness and the extent of their learning through increasing student-student and faculty-student 
interactions (39). 

Project results: In accomplishing these objectives, we validated 47 case-based 

videotapes, and canine and ruminant KBIT analyses of neurotoxicant space. We disseminated 
of our learning system throughout our college and into two other North American Veterinary 
Colleges. Academicians from Mexico, Brazil and other Pan American Veterinary Colleges have 
expressed interest in the possibility of translating our case-based scenarios which apply to their 
unique areas. Visually enhanced stories were filmed and produced as case-based videotapes. 
These tapes were used as the framework for advanced learning in junior clinical toxicology and 
mentoring clinical toxicologic medicine during their senior rotation. Groups improved student 
involvement and intragroup cooperation. Computer programs provided the means of deciding 
the essential features defining the clinical disease and differentiating this disease from other 
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similar diseases. This knowledge was evaluated using cases just like those encountered in 
veterinary practice. Students began to be able to tell just how good they really were-and take 
credit for their accomplishments-by evaluating their knowledge bases against a battery of 
clinical cases (KBIT; previously experienced as junior groups; 20-30% of the 1994 graduating 
class) and by answering randomly drawn "quiz-bowl" types of questions on their senior 
rotations, prior to taking their state board licensing examinations (10% + students from 1995 and 
1996 graduating classes). This year, synopses of teaching cases placed on reserve after initial 
examinations (1995 Junior Class Clinical Toxicology), furthered this feature of learning. 

Initial attempts to use group hourly evaluations met with frustration from students, 
because they could not arrive at a consensus for their examination questions, although students 
appeared at least as strong in clinical rotation as in previous years. This fall at Kansas State 
University, we made a second attempt and observed enhanced team performance on examination 
when compared to individual examination performance. We divided our class of 89 clinical 
toxicology students into learning groups of five to six individuals each. One third of the groups 
were quizzed as groups, while the remaining two-thirds of the students took the quizzes as 
individuals. The students responding in groups could recommend better treatments (p <_ 0.05), 
develop better diagnostic reasoning (p <_ 0.05), and explain better mechanisms (p <_ 0.05) of 
disease than could the remaining 2/3 of the students learning as groups but taking quizzes as 
individuals (93a). Although the group experience and its feedback improved students’ ability 
to explain mechanisms of disease on a subsequent individual examination (p <. 0.05), it did not 
lead to better overall examination performance, suggesting the strengths and limitations of future 
professional consultations. Students validated one anothers’ recommendations for treatment, 
diagnostic hypotheses, and mechanistic explanations, representing beginning expertise-mastery 
(93a). By the second hour examination, groups had more treatment completeness and greater 
depth of explanations of mechanisms of clinical signs (p<0.05 Student T) and increased test 
scores 1.5 weeks after the initial quiz. We hypothesize that additional steps toward expertise 
have been shown. The group experience enhances the adventure of difficult explanations, and 
that is most readily seen after some development, but proximate to the initial experience. 

Transition to limited, temporary, expertise was noted on oral examination. Third-year 
veterinary medical students’ oral examination commenced with students stating a species 
preference, drawing a case from a card-deck of one-paragraph cases and continued by them 
analyzing the case. Criterion for full credit was a working knowledge of those given for 
intergroup analysis in large-group problem-based learning. Students ^showing expertise equal to 
that expected in practice or beyond that of the instructor were noted ( ). When selected students 
so graded were later interviewed, most were not aware of the quality of their performance. We 
have compared the transition rate of students taking no previous oral examination (7%; 3 
times) to those undergoing a voluntary practice examination (in 1-week before examination 34 
accessions from 18 students)(10%), 1 previous examination (11%) and 2 previous oral 
examinations (23%). Practice or the challenge of taking one or more previous examinations 
increased the rate of transition to temporary, limited, expertise. 

Initial analyses of a KBIT small animal neurotoxicological problem-space will continue. 
Individual students (fourth-year) were compared to groups of 3-4 students (third-year) and to 
students working both as groups (third-year) and as individuals (fourth-year). Compared to 
beginning individuals, our groups modestly validated each others strengths (common toxicants 
+54%; PNS), but failed to compensate for each others weaknesses (less common toxicants - 
22%; p _<_ 0. 1 student T). Students with both team and individual experience compensated each 
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others weaknesses (+116% increase vs -22 % ; p <. 0.01) effectively. KBIT allowed the students 
to clearly perceive their weaknesses. 

Evaluation/Project Results 

APPROACH: We proposed that evaluation procedures for this project be flexible and 
responsive to the project goals, activities, participants, and staff. Our change of chief evaluator 
from Dr Larry Enochs, John Staver (and W Pallett, V Clegg; 1st project year) to David Balk 
(and W Pallett, V Clegg; 2nd, 3rd and 4th project years) made this an especially pertinent 
proposal. The design took the form of the widely accepted Stake Responsive Model (Stake, 
1975), that called for a multiplicity of data sources, including both qualitative and quantitative 
data. ' Our data include qualitative and quantitative data, triangulation of qualitative with 
quantitative data, and triangulation of both qualitative and quantitative data with investigator 
observations. A strength of this model, is that we will continue to define relationships which will 
further the aims of this project, and to improve case-based learning. This heuristic model led 
to better understanding of LG-PBL, SG-PBL and visual learning. Tentative working hypotheses, 
listed below, formed expectations subject to reformulation as the project proceeded. The 
discovery of new relationships, concepts, and understanding resulted. In addition, possible 
verification of the tentative working hypotheses emerged. The heuristic nature of the model led 
to such tangible payoffs as: 1) explanations of reasons for problems, the background of a 

situation, what happened, and why; 2) explanations of why the program worked or failed; and 
3) an analysis of its applicability. In addition, this model was flexible enough to ascertain 
student gains on traditional measures and factor those results into resultant interpretations. 
Evaluations were depicted in the form of case study scenarios along with more traditional 
quantitative reports. 

The process that was designed to evaluate the effectiveness of the enhanced LG-PBL 
program has two objectives: 1) to determine the impact of enhanced LG-PBL for students and 
for teachers (summative measures to assess improvement of student learning), and 2) to provide 
a basis for improving the program itself (formative measures to assess the program’s success). 
To accomplish these objectives, several types of information, both quantitative and qualitative, 
were gathered and analyzed. Evaluation techniques include written and oral examinations, 
reports, and observations. In addition, regularly scheduled interviews of both students and 
faculty will be utilized. 

Students from each of the 4 years of Veterinary program were sampled by pre-formed 
instruments, observation (78), videotapes (98, 99) or interview. In addition, we sampled 
Doctors of Veterinary Medicine in internships, residencies and as faculty (40). 

A variety of evaluation techniques were used. Instructors established clear standards to 
guide in-class assessments (written examinations, oral evaluations) that determined how well 
students are accomplished the learning objectives in each course. These assessments were an 
inherent component in the instructional procedures of each course. Students, LG-PBL 
instructors, and professional staff on the clinical floor of the KSU Veterinary Teaching Hospital 
as well as at sites of clinical internships were surveyed regarding their evaluations of student 
progress toward the objectives listed above. Classes were videotaped and observed by a faculty 
development specialist so that student learning in the instructional situation could be carefully 
analyzed to determine the correlation to the established instructional objectives (8). 

The assessments described above were be made during the project’s four years (and will 
be made at a reduced level in subsequent years). During project months 8-18, long-term 
evaluation strategies were developed. These involved collecting several types of information 
from those who were in their clinical year. Among the information to be collected was: 1) 
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descriptions of confidence as pre-clinicians; 2) retrospective reactions to various facets of the 
enhanced LG-PBL program; and 3) supervisory ratings of how successfully 4th year students 
fulfilled various responsibilities. 

In this project we tested several hypotheses relating to specific outcomes. 

1) Practit ioner /Faculty-generated cases increased stud ent involvement. Visual 
enhancement will increase it still further. Involvement was measured as student-student and 
student-faculty interaction by randomly sampling videotapes taken throughout the course to 
document the enrichment of LG-PBL. Validity of sampling such tapes was suggested by Patton 
(98). In addition, interviews of purposively sampled and volunteer students was conducted. The 
same students were used throughout the course to ascertain trends and emerging data. These 
interviews will be transcribed and analyzed by way of content analysis. The evaluators 
interpreted these resultant analyses and generated assertions. The interviewees assisted in the 
analysis to corroborate the findings. 

2) The enhanced cases altered student behavior from listening and repeating and relating 
t o analyzing and creating and evaluating. Change in behavior was detected by direct expert 
observation of videotapes sampled at random from classes using enhanced LG-PBL (98). The 
evaluation process determined the degree to which students develop higher order cognitive 
processes essential to diagnostic and decision-making abilities, e.g., applying ideas in new 
situations, analyzing to discover hidden meaning and to distinguish between fact and opinion, 
and formulating distinct criteria on which to base judgments. 

3 ) The enriched cases led to qualitatively greater dep th of performance. Cooperative 
learning will e nhance it still further. This was demonstrated as the relative incidence of 
practitioner quality answers during oral evaluations. One of the coinvestigators (JAP) has been 
recording the occurrence of such answers as part of the oral examinations in Clinical 
Toxicology. This is a qualitative, observational measure of temporary transition from "pre- 
expert" to "expert." In an LG-PBL class not previously given such examinations, the incidence 
is 7%. This incidence increases to 11% when the class have all had one previous oral 
examination, and to 23% when they have had two examinations. Interviews shed light on this 
hypothesis. Readiness for professional practice was assessed not only as a function of basic 
knowledge and understanding and decision-making ability but also as a function of important 
professional skills such as effective communication, the ability to interact with clients and 
professional colleagues in new settings, and the ability to continue learning independently. 

4 ) Modified K-BIT accurately discriminated between low and high levels of diagnostic 
skill bv using an exnert panel of practitioners. Expert practitioners (practicing clinicians) formed 
the benchmark of performance against which knowledge bases are judged. Moreover, the 
percentage of the case battery which such experts correctly diagnose was higher than with the 
same novices. A reflection of their diagnostic maturity was dependent on precise definers of the 
model, as opposed to discriminators from similar diseases anticipated for novices. 

5 ) Linking the greater propo rtion of essential visual and writte n detail from enhanced 
case formation to computer software, enriching si gnificant content of student knowledge bases 
led to enhanced performance. KBIT profile were cross linked to round performance in senior 

rotation, but no clear trends have as yet emerged. 

6) Differences in pathway reflected differences in diagnostic maturity (relative degree 
of clinical experience! and different clinical decision making style. KBIT software did not allow 
testing of this hypothesis at this time. 

7) Because gf thjs trkL Q least Qi the groups gf six students tgok credit for pre^ 
expert to expert transition. Pre-expert to expert transition was demonstrated as 
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unusual depth of performance (see test of hypothesis #3) and an increase in diagnostic maturity. 
That students took credit for the transition was validated by student survey of Junior Clinical 
Toxicology. Additional validity was gained by way of interviews (see hypothesis #1). PBL did 
bolster self confidence. 

RESULTS: During the first project year, we focused on standardizing our evaluation 
technique and team. We detected novice to limited expert transitions and expanded that 
measurement. During the second project year, D Balk agreed to head the evaluation team, and 
we initiated qualitative and quantitative surveys about the nature of PBL. From this several 
widely held beliefs (themes) and hotly contested issues were confirmed. We evaluated use of 
LG-PBL in the first portion of the toxicology course, and placed it in the context of what it was 
like to be a 3rd year veterinary student. We evaluated the confirmation or lack of confirmation 
of our hypotheses for this project. Teaching partners (CE Layton, JA Pickrell) is included 
because it provides additional means of triangulation of data collected from the formal 
evaluations, and to provide experience for investigators in ongoing in classroom evaluations. 

Teaching partners, a partnership between 2 teachers of roughly equal expertise in which 
we visit each others classes 3 times and interview 3 students) excited veterinary students because 
faculty effort was directed toward improving their education and their opinion had been asked. 
Students revealed a bias toward action (hands-on experience), regarding it as their best 
learning experience. Our evaluators correctly raise the issue that too much of this good thing 
could lead to students becoming technicians of Veterinary Medicine, rather than true 
practitioners. The teaching partners take our responsibility to guide them so that this does not 
happen seriously. Students increasingly used groups for consultation throughout their 
education, learning in groups, and as their predominant method of study. Our previous 
evaluator surveys reveal that some form of lecture and problem-based learning is 
overwhelmingly preferred as security to validate their learning. As partners, we point out that 
this shortens intragroup/team learning. An alternative to this apparent paradox is to test in a 
group. If this is always done, the novelty quickly wears off, and teams have trouble agreeing. 
This fall preliminary data (see above) was gathered where we alternated between testing in 
groups and individually, teams lead individuals to significantly increased competence in 
treatment, diagnostic reasoning and explanation of clinical signs, leading occasionally to 
increased competence on individual examinations. Students especially appreciated the support 
provided by these teaching partners, although many felt it to be instructor dependent with respect 
to their other instructors. Students really liked individual oral examinations, especially those 
in which questions of ordered difficulty were given, supporting the examination’s positive 
outcome. Our preliminary data suggests minimum time/student should be 10 minutes. Our 
evaluators correctly point out that examining groups of 4 students in 15 minutes suggested the 
need for more time, although students were clearly involved. Most (71%) of the students felt 
that examinations in the partners courses accurately measured their knowledge. 

Summary form was judged to provide better understanding than graphs. 

Themes (widespread beliefs) uncovered by interviews were (those confirmed are 
italicized and underlined. Investigator comments are italicized): 

• Students are responsible for their learning ( but they resist this). 

• PBL works when objectives are clear. 

• Evaluating effects of PBL is a shared faculty concern. 

• The College of Veterinary Medicine instills and promotes student fear of faculty {or a 
fear of not measuring up). Professional year students in years 1-3 express a need for 
perfection. Maturity allows increased perception of reality. 

• Relevance of information should drive curriculum decisions 
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• Students learn more once they enter veterinary practice. Students take credit for 
achievements with difficulty until after they graduate. This belief may be an illusion of 
structuring in practice as they encounter problems. The survey was conducted too early 
to see any effects of PBL. 

• Groups provide a break from lectures (our surveys show this too). 

• PBL takes a lot of time ( although it doesn’t have to be that way; confirmed) . 

• Ambiguity is inherent in PBL (confirmed) . 

• Facilitators are the key to student success in small-group problem-based learning 
(PBL) (gpnfu med). 

Issues dividing stakeholders over PBL (those confirmed are italicized and underlined): 

• The administration of the College of Veterinary Medicine is committed to PBL. 

• Prior knowledge is needed for students to profit from PBL. 

• Some courses are not appropriate for PBL 

• The meaning of PBL is a concept shared by all in the College of Veterinary Medicine. 

• Combining lectures and PBL would be more effective than running lectures or groups 
by themselves (confirmed) . 

• PBL builds student confidence (confirmed) . 

• PBL increases retention of information (confirmed) . 

• PBL promotes in-depth learning at the expense of breadth of coverage (confirmed) . 

In addition, Dr. Balks team confirmed that Learning the process of problem solving is 
a goal of PBL. 

Evaluation of the Toxicology Course, indicated partial validation for the following 
objectives (investigator comments are italicized): 

• Faculty provided a structure which will allow students to integrate what they have learned 
and enhance student transition to temporary experts. We present quantitative evidence 
of such transition on oral examinations. However our evaluators correctly point out that 
students present powerful resistance for being responsible for their own learning. 

• Faculty provided exams which foster a colleague to colleague, not instructor to pupil 
relationship in order to facilitate learning. We did foster an instructor-dependent 
colleague to colleague relationship. 

• Faculty developed tests which reflect the ability of the students. 71 % of students from 
the teaching partner survey conducted after students had received their grades felt that 
exams fairly reflected their ability. However, students arguments indicated expectations 
of coverage only of material in class, but were held to material assigned. Arguing with 
referees at sports contests come to mind. 

• Students changed learning strategies from memorization to learning throughout their 
professional careers. Students understood basic mechanisms, preferred to have them 
explained by the instructor and preferred to look up details, much like group consultation 
in practice. 

• Education was direct instead of student self-study after lecture, empowering and 
challenging the student through the use of real-life problems. In one course of 7, 
students prepared and discussed cases well until mid-semester when the increased time 
commitments became overwhelming, and the instructor assumed more responsibility for 
their learning. 
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Case studies of 3rd year veterinary students revealed the following (investigator 
comments are italicized): 

• Students receive large amounts of information causing them to fall behind throughout the 
year. From mid-semester students do only enough to receive their target grade (typically 
an A or B). It is much harder to implement PBL in the last half of the semester, and 
almost impossible to do it in isolation (1 course in 7). 

• Student’s lives reflect their scheduled hourly tests (0, 1 or 2) for the week (at odds with 
next). 

• Students use their spare time to do things that they enjoy, that is they don’t study all the 
time (at odds with previous)(yet this report provides examples of giving up cherished 
activities; these decisions reflects decisions needed in most practice lives; and students 
really love veterinary medicine ). 

• An A or a B is an acceptable grade for a 3rd year veterinary student (this is at least 
regionwide, but limited preliminary data suggests that these goals are not achieved). 

• Groups serve an important function in the lives of veterinary students. They rely on and 
support each other— in laboratories and in other ways. Our preliminary data says that 
group and individual testing can help students learn more effectively. 

• The 3rd professional year clearly marks the transition between the first 2 years of basic 
science and the clinical year (4th year) before students enter their practice life. Students 
hold to what has allowed them to survive and prosper, and resist activities pushing them 
toward their senior year, while indicating a desire to get into the clinics. As faculty, we 
must help them make this transition ( see next point). 

• The main concern of 3rd year professional students is preparation for the 4th year. 3rd 
year students fear (or question their learning or measuring up to) the mysterious 4th year 
rotations (see previous point). As faculty, we tell them to investigate further, nearly all 
of their predecessors have survived and are prospering in practice, so can they. This 
generally dispels their anxiety. 

• Students do not believe the national board is an effective method of testing their skills 
as practicing veterinarians, they favored their ability to apply skills learned (not too 
different from the apprenticeship model of days gone by). Students do need to be licensed 
to practice, we must help them achieve their dreams. 

• Students showed signs of burnout during their 3rd professional year (possibly the burnout 
[unfinished business] is a function of the large amounts of information and the impossibly 
high self imposed standards they experience). 

• Students respect faculty who treat them with courtesy (willing to help outside of class; 

grade exams promptly; treat them like professional colleagues) and who have significant 
practical experience (knowledgeable; provide practical information in a way that they can 
understand). Our practitioner- generated, case-based videotapes were well received 

because of this. 

Our evaluators and investigators validated the following hypotheses during this project. 
c A SE B A SEP VIDEOTAPES : Practitioner-generated cases and case-based videotapes 
did increase student involvement (Hypotheses ill and 2) as estimated by instructor (55-85%), 
observer (>15%) and by videotapes of selected lectures ( > 90%). Our evaluators point out that 
in small groups, involvement is facilitator dependent, a fact borne out by literature on PBL. 
Case-based learning did increase the incidence of practitioner quality answers on oral 
examinations (Hypothesis #3), and groups supported students and promoted interaction. 
However, a minimum of 10 minutes was required for each student, whether in a group or not. 



Our evaluators note correctly that 15 minutes/group of 4 is insufficient to have students perceive 
themselves as being evaluated completely. At least 80% of the students did ngt take credit 
for transition from novice to limited pre-expert ( Hypothesis 47). Either our estimate was 
optimistic, or our methods insensitive. We find that validation of learning is difficult to get 
students to admit and estimate the rate at 10-60%, increasing with supportiveness of instructor, 
practicality of conditions and maturity of student. 

KNOWI FDCF BIT INFORMATION TRANSFER PROFILES : KBIT computer 
profiles of the neurotoxicant problem space in small animals could distinguish mastery 
levels of diagnostic skills from low levels of diagnostic skills {Hypotheses 44 and 45) as 
indicated by comparing senior skills near performance expected for practice with enhanced skills 
of students first analyzing the profile as a team of 3-4 juniors and subsequently as individual 
seniors (data reported under KBIT neurotoxicological profile analysis). Our experience suggests 
that because of its complexity of diagnostic reasoning by computer, the profile underestimates 
diagnostic skill of students with low to moderate diagnostic skill. As students approach mastery, 
progress appears to be quite rapid (94, 95). Students more easily identify areas which need 
work, and had more difficulty taking credit for their achievements. The 6th hypothesis, that 
students with high diagnostic efficacy would have a different diagnostic path was not testable 
with current KBIT computer software which could not track more than an initial and a final 
answer. Our preliminary data suggesting that group examinations drive treatment and 
explanatory inventiveness suggest the scientific validity of this hypothesis. 

Dissemination: KSU Institutionalization— Residents and faculty participated in the making and 
using of these videotapes. Several sections use related tapes, their tapes as part of their senior 
rotation. Nation/international dissemination— 12 videotapes are in 2 colleges of veterinary 
medicine, University of Texas (College Station, TX) and Atlantic Veterinary College (Prince 
Edward Island, Canada). Numerous presentations have been made throughout the veterinary and 
toxicology professions demonstrating tapes, their utility and future directions of this research. 
Practitioners at our annual conference, students who are graduating and veterinarians from 
Mexico, Brazil and other panAmerican countries (translation required) have expressed interest 
in our case-based videotapes to model clinical conditions. Dissemination of videotapes 
continues. 

Residual benefits from the project: Students learned the richness and variety of practice cases 
associated with toxicology, many aspects of veterinary medicine. The students interfaced with 
practitioners who will have a chance to see just how expert they are becoming in their 
differential diagnoses and problem solving as indicated by our increasingly active externship 
program. Faculty achieved satisfaction in enhanced student learning and practice expertise 
deriving from LG-PBL. However, they experienced misgivings about the perceived lack of 
breadth of coverage as they compared their students achievements with those of more 
conventionally taught courses. Such anxieties are common in the medical profession for both 
students and faculty. The heavy evaluation emphasis aided faculty in forming their teaching 
portfolios, increasing their value throughout professional education. Researchers with little 
teaching experience were brought up to speed to use PBL in academia. This flexibility in 
recruitment will make it more likely that recruited faculty will form centers of excellence for 
research and to lead post graduate or post doctoral students. The first such center at KSU 
College of Veterinary Medicine currently focuses on beef production medicine and animal 
science. Future plans and funding include leading of graduate or post doctoral students. 
Practitioners and their clients are the ultimate beneficiaries of improved training, clinical acuity 
and practice efficiency in our graduates. Tomorrow’s practitioners will be challenged by the 



increasingly complex problems of veterinary medicine, most of which they had no role in 
creating. Closer liaison with veterinary educators and specialists allows improvement of 
continuing education and provides data upon which improved clinical decision-making skills can 
grow as well as making more efficient use of tertiary health care in veterinary medical centers. 
Our vision of the future includes further definition of the path to expertise. Currently, one 
investigator (JAP) tests in groups and individually and have demonstrated increased effectiveness 
of treatment, diagnostic reasoning and ability to explain mechanisms behind development of 
clinical signs on group efforts. Students also show increased ability to explain mechanisms on 
individual tests, and greater willingness to invent better treatments and explanations as indicated 
by their participation in these difficult questions on an hourly examination. We believe these 
activities represent initial steps on the path to expertise. We believe that they are reinforced by 
review of our case-based videotapes. Finally, we believe that they can be detected using the 
KBIT profile of F Papa (TX Coll Osteopath Med, Ft Worth, TX) and his simultaneous 
measurement of knowledge base with varying case atypicality. 

Summary and Conclusions: Forty-seven case-based videotapes were produced and are used 
to model clinical conditions at KSU College of Veterinary Medicine, increasing student 
involvement and the quality of their learning. Learning appears greater if oral examination 
questions are ordered from simple to complex. The neurological problem-space of 3rd year 
veterinary students was evaluated most successfully in initial groups and subsequently as 
individuals, distinguishing skilled from novice diagnostic reasoning. Our evaluation team and 
investigator triangulated findings reveal that PBL fosters significant advantages, increased student 
confidence and in-depth learning. However, PBL’s significant disadvantages, reduced breadth 
of learning, considerable time requirements, and significant ambiguity while learning are 
regarded as barriers to introducing PBL curriculum wide in many higher educational institutions. 
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RE: Response to evaluation of Santos, Susan and Balk, David E, Evaluator analysis 
of interviews conducted by John Pickrell ( and Candace Layton), July 1995, Received 

7/31/95. 

INTRODUCTION 

This study was conducted paralell to the FIPSE study, but had some results 
which bear directly on those of our evaluators, adding new information and presenting 
at least one opportunity for triangulation. Teaching Partners are neither common, nor 
especially rare throughout the country. To our knowledge, this partnership was unique 
in coinvestigators of this study. Additionally, by asking general questions it provided 
general information about our student population which was of interest to our 
coinvestigators. Finally, I’m not certain that all the features of this unique program 
are immediately apparent. This response presents additional information for our 
evaluators to consider. As always, we welcome additional responses in synthesizing 
our final report. 

METHODS 

The analysis is of formatted interviews from principally Junior and Senior 
Veterinary Students as part of a Teaching Partners project. Teaching Partners is 
inspired by Joseph Katz’s program for New Jersey Master Teachers (Katz J, Henry 
M, 1993. Turning professors into teachers. Oryx Press, Phoenix, AZ 173 p.) and 
expanded throughout the US, mostly by Steve Golin. Dr. Layton, a soft tissue 
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surgeon and 1 first encountered it at a Kansas City Regional Council for Higher 
Education (KCRCHE) of which Kansas State University is a member. I presented 
some work in PBL at a later meeting for KCRCHE and at a KCRCHE Master 
Teachers Workshop. Teaching partners have these goals. 

[ Find your teaching partner, bond based on the mutual goals of 
improving your teaching. Teachers of approximately equal abilities 
will find the pairing more productive. Teachers should be at the same 
or a close institution, but need not be in the same content area. 

2. Prior to class, find out what your partners goals for that session are 
and use class response to indicate what was going on. Your partner 
must take your, her/his data and decide if these goals were met. 

3. Attend 3 of each others class meetings, lectures, discussions or case 
analyses. 

4. Interview at least 3 of your partners students and ask them what your 
partner could do to improve her/his teaching. 

We may constitute a unique use of this program since we shared the same 
students. Thus, both interviewees interviewed their own students. We suggested to 
our students that they be interviewed by the person they wanted to correct the least, 
so that they would feel most at ease. Most students did not seem to mind, because 
they were talking to their instructors, who were spending their time with them in what 
must have seemed to the students to be a serious effort to cocreate a new course. In 
addition, neither Layton or Pickrell were regarded as being especially threatening. 
Finally, their didactic instruction by Layton and Pickrell were completed and they 
were less nervous. 

Secondly, the students were interviewed at the end of the course, when 
midcourse corrections were no longer possible. Data was valuable supplement to 
written TEVAL evaluations for any potential changes in the course. Katz and Golin 
make it clear that more benefit would be derived by interviewing the students during 
the time of the class. Certainly, the detail would be clearer, but students would lack 
the perspective of those successfully completing the course. We continue to consider 
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how best to implement this (Dr. Layton has returned to private surgical specialty 
practice; Dr. William Fortney is partnered with Dr. Pickrell). 

As summarized by Susan Santos, the responses were coded and categorized by 
themes. The answers had multiple components for some students (the evaluators 
assumed that the questions had the same components). Actually, our evaluators 
instructed us to ask the questions in open-ended manners so that we elicit as much 
information as possible. The interviewers tended to allow students to skip and 
subsequently eliminated questions for which uninformative or no answers appeared to 
be emerging. The multiple components of student answers were so informative that 
they were formalized into questions for later surveys. 

The survey is divided into 2 sections, general, applying to veterinary school 
and special, applying directly to the courses of Layton and Pickrell. In some cases, 
the answers to general questions about veterinary college would define several 
populations who might have responded differently. Our observations are that the 
response to general questions had little to do with responses to specific questions One 
must remember that these students were chosen randomly from students willing to 
participate (usually > 50% of students). We owe students a debt for being willing 
to take their time and share their views with us their instructors. 



RESULTS AND DISCUSSION 

PRACTICAL EXPERIENCES: Students knew early in life that they wanted to 

attend veterinary school (52%), because they like animals (23%) and wanted to be a 
part of the community (10%). Practical (hands-on, real-world) experiences were 
most likely to verify that students expectations for veterinary school had been met 
(most common response; 28%), were regarded as something which should be done 
in instructing them, reflected what they felt to be their best learning style. Half of the 
students regarded hands-on experience as their best learning experience. They felt 
case-based (problem-based) learning (toxicology and surgery; intermediate response 
14%), visual instruction or surgery and videotapes (both) were good substitutes for 
hands-on learning. Two factors are apparent from this analysis: Although practical 
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learning was popular and regarded a strength of this teaching partnership, occasional 
students wanted still more practical, hands-on or case-based learning. Secondly, had 
students been allowed to design their own experiences, they would have been very 
heavy on practical or hands-on and weak on theory, content or medicine and surgery. 

Clearly from their comments, and the comments of our evaluators (Susan 
Santos and David Balk) students may not understand the utility of theory. We note 
that alternatively, the students may believe that if they work side by side with 
someone who understands more theory than they would ever want to know that the 
conversations with those clinicians will somehow rub off and be incorporated into 
their knowledge bases. In this case, they’d probably like to see how the specialist 
clinician uses that theory. (We note, that this line of reasoning closely approximates 
the justification for apprenticing that was used prior to education in veterinary 
colleges.) This survey does not distinguish between these alternatives. The evaluators 
Susan Santos and David Balk have discussed the first one at most length and we 
partially support their reasoning regarding these student recommendations. Our 
veterinary college as a whole believes that students need both. Thus, we give them 
3 years of mostly didactic lecture-based instruction and 1 year of clinical instruction. 
Our licensing examinations reflect this division of education by requiring them to pass 
both a national board examination, based mostly on their didactic instruction and a 
clinical competency test. The pass rate nationally and at Kansas State University is 
very nearly equal for both tests at mid senior year (>80%). It is clear that the issue 
of theory must be incorporated into practical veterinary medical-medical experiences 
throughout the professional education. Our study is one of several (Medicine: 
Albanese M, Mitchell S, 1993. Academic Medicine 68: 52-81; Barrows HS et al., 
1986. Med Teacher 8: 325-331; McAuley RG, Woodward CW, 1984. J Med Educat 
59: 842-843; Normal GR, Schmidt HG, 1992. Acad Med 67: 557-565; Wilkerson 
L et al., 1991. Acad Med 66: S79-S81; Veterinary Medicine: Cornell 
Program— Cornell Veterinarian; Mississippi State Program and others— PBL in 
Veterinary Medical Education: Challenges in Implementation— March 2-4, 1995, San 
Diego, CA) viewing the problem from different perspectives. To my knowledge, the 
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challenge of balancing practical hands-on training and theory in the medical field 
remains to be solved. 

Theory, or a complete understanding of mechanisms is frequently of use in 
cases which are especially difficult or complicated, or in cases which there is as yet 
no adequate therapy or surgical correction. The art of practice is replete with the 
values, insights and concepts which help students organize the vast amounts of new 
knowledge which pours into them every day of their professional lives. K Patricia 
Cross has said, "Education is what remains long after you’ve forgotten the specific 
details". Students agreed with this statement (15/15; 100%), and provided examples 
that indicate they are beginning to understand that theirs is more than just a technical 
education. Numerous direct quotations in this summary indicated that they have faith 
in their instructors, specifically these teaching partners giving them the technique, art 
and values of practice. Alvin and Heidi Toffler tell us that our knowledge 
architecture (capacity; what you know and what you can do with it) is our graduate’s 
most salable product and that diversity will drive our future accomplishments. They 
say further the progressive apprenticing (our equivalent is hands-on training) will do 
far more good that money or labor in building our country’s (we read profession’s) 
achievements. 

GROUPS: Our veterinary graduates will be involved increasingly in team consulting 
and cocreating optimized health care for their patients. Frequently (25%; most 
frequent) of veterinary students surveyed recognize that some diagnostic and 
management problems can be of incredible complexity and difficulty and have already 
begun to use consultation to surmount those difficulties. Most felt any information 
about how to solve difficult or complex problems would be a valid addition to our 
courses. A larger number learn in groups, a form of intrastudent consultation— 21 % 
identified it as their best learning style. 30% (most frequent response) use groups as 
their predominant method of study, while an equal number studied alone. Students 
tell us that groups allow students to support and help each other with their most 
difficult assignments. Interactive learning, where students and instructors partner 



BEST COPY AVAILABLE 



briefly, was frequently cited as helping students learn in toxicology and surgery 
(41%), as opposed to good outlines (14%), written cases (14%), minilectures (14%) 
and "not talking down to them" (10%). In fact, 18% of students (5/28) asked that we 
place important or common toxicant/surgery cases in the library so that they could 
study them. 

Team learning is thought to be the one of most powerful learning currently 
being practiced (Senge, P et al. 1994. The fifth discipline fieldbook. Currency, 
Doubleday, NY). In Veterinary Medicine and Medicine we practice this as group 
study from a variety of classes, cooperative learning, lecture-, case- and problem- 
based learning, and small group problem-based learning. Teams are formed with 
mutual respect, and discussion of growing skill and some dialogue or brainstorming 
is used to learn subject material which would have been more difficult individually. 
Students have growing skill in expressing their complex results. Advantages of this 
technique are thought to foster continuation of study groups. The structure of small 
group problem-based learning supports most of these steps. Although groups are 
randomly assigned (as in cooperative learning), in many cases, because of intensely 
successful working conditions, profound respect and new contacts develop. By way 
of comparison, the orchestration and shortened intragroup discussions of large-group 
problem-based learning may reduce consistent development of skill in discussion, 
brainstorming and dialog, although some groups do acquire most skills and learning. 
Hybrid lecture-large group problem-based learning preferred by a previous 
evaluator survey will shorten intragroup discussion still further, but summarize 
student-instructor interactive findings more completely. 

SUPPORT: Students especially appreciated the support provided them by Layton 
and Pickrell (7/7; 100%), especially in not being intimidating and supporting them 
even when correcting them. All students (7/7; 100%) felt they would like to be 
rescued without being intimidated. Students cited the support as the most important 
feature meeting their goals for the course with intermediate frequency (8%). Nurture 
helped build their confidence and more deeply involve them in the processes of 
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diagnoses and management and to regard their relationship with professors as one of 
partnership (14/25; 56%) as opposed to ambivalence (24%), antagonism (8%) or 
disinterest (4%). Many indicated that this was somewhat instructor dependent. 
Students frequently (9/17; 53%) cited being treated with respect, that the instructors 
really cared about them as what they liked best about the course. This support may 
be a factor in the high approval rate of this teaching partnership (89%) where we were 
trying to improve their teaching and actually asked their opinions. Several students 
have indicated that beginning the senior clinical year, where it all comes together, 
is the most rewarding and yet the most stressful event of their veterinary medical 
education. About half (49%) indicate that they expect this to be a positive response. 
Encouragingly, an additional 14% were apprehensive, but now regard it positively. 
Approximately 1/3 remain apprehensive, but expect to survive. 

Nurturing (safety), education (stability) and building (precision) are 3 major 
components of order. These are often opposed by chaos (self determination, liberty 
and non-conformity [anti-cause and e t fee t [ ) . When safety and stability are sufficiently 
high, one can often ignore them momentarily and violate cause and effect to solve a 
problems. Without this, problems are solved much more hesitantly less building 
occurs (Donaldson S, 1991. Forbidden Knowledge , Bantam, NY). Thus, nurture is 
necessary for rapid student progress. Our impression is that students answer more 
difficult problem solutions if they feel safe and if we ask them easier questions first, 
to prime the pump. This appears to work as well with content as with problem 
solving questions. Students mention the positive effect of being treated with respect. 
This is a vital component of team learning (see above), and the instructor and each 
student may be regarded as simple teams. 

CONTENT QUANTITY: Students felt that there was a great deal of material 
(content; 27%) and that it was difficult to keep up (69%). The veterinary medical 
curriculum is constantly expanding and what material to use is a constant 
consideration. Students suggested that the material they should learn should be 
reduced. They felt that being taught only the common toxicants (41%) or the most 
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common surgical procedures (46%) would simplify their lives; we suggest that the 
simplification might be temporary and that it must be carefully conceived and 
executed. For example, we experimented in 1994 fall with a hierarchial organization 
with mixed results (McKay, Toxicology during the fall semester, 1994: The fifth 
evaluation of the FIPSE-funded innovation in the college of Veterinary Medicine, 
1995). Only 1 of 2 instructors followed the proposed modification, the one who wrote 
it. This led to student perception of 2 different approaches to toxicology, generating 
significant anxiety, not present prior to this modification. More than half (74%) of 
the students wanted to learn to diagnose the need for common surgery better. Finally, 
students indicate knowing the common diagnoses uncommonly well, especially 
including the common differential diagnoses were invaluable to them. 

EXAMINATIONS: Most students believed that the purpose of an examination was 
to find out if the students were where they were supposed to be with respect to course 
material or to assess what the students had learned. One perceptive student felt that 
instructors and students would both learn by the examination; we agree. Another felt 
that they reflected back on us— that we were known by their quality; we agree. 
Neither I or the evaluators noted that students thought they could learn by 
examinations. However, 1 (JAP) see some evidence of this. For example, students 
check their progress (daily, course, year and veterinary practice) by self tests (50%) 
and official examinations (36%), although they are somewhat ambivalent about the 
latter. Secondly, they favor oral examinations (35% would increase their number in 
the examination schedule [most frequent response]; 35% felt that Dr. Layton should 
use them like Dr. Pickrell did [most frequent response]; 12% liked them best about 
the 2 courses and 11% suggested their increase as a curriculum revision [responses 
of intermediate frequency)). Students tied their preference for oral examinations to 
their ability to learn from them and integrate the information that they were already 
taught. We believe that the stated factor was important. However, we have recently 
learned that undergraduate students believe grades should include some effort 
component and are at least partially negotiable (Goulden et al., 1995, Kansas State 
University). Oral examinations include more than just the correct answer. They feel 
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that their score reflects not only their answer, but what they appear to be able to do 
with it (Knowledge Architecture; ie., hesitancy, practicality, and reasoning). It is like 
their experiences when interviewing as seniors. It is our impression that students feel 
that they at least partially negotiate their oral examination grade and that that 
perception is important to them, more important than it is to their instructors. 
Students frequently commented that instructors and students should both learn and 
teach in the course (62%), as opposed to the more traditional view, that students learn 
and instructors teach only (15%). They felt that they had learned, and occasionally 
cited examples where they felt that instructors had learned both from the course and 
from the examinations, generally the oral examinations. 

Faculty perception of the purpose of an examination is to illustrate that students 
knowledge base relative to the questions asked; most students perceived this. If those 
questions relate to the desired knowledge base and to the real world, interesting events 
take place.. Instructors are perceived as asking about questions highly relevant to 
those the student will need to answer. Secondly, instructors are perceived as asking 
questions which accurately reflect students knowledge. Thirdly, instructors learn 
about the students knowledge base— how well their concepts are understood and can 
be applied by students. Finally, instructors learn— often a great deal— about the 
content of the areas on which they ask the questions. Thus, the instructors learn a 
great deal by each examination which is rooted solidly in their course content and in 
the real world. 

Students also learn. They learn how to respond to real-world questions, and 
how their expert instructors expect students to respond to those questions. They learn 
what they know and what they need to know about these questions to perform in the 
real-world. If exams are conceived of appropriate difficulty, they learn that they have 
learned the material very well and can apply it to a real-world situation. Thus, 
students learn a great deal by examinations. Students in this survey recognized that 
exams are a learning situation for both faculty and students, but did not provide 
further details to illustrate these points. Their instinctive preference for oral 
examinations suggested that on a different level they understood the value of learning 
from examinations. Their ambiguity about using formal examinations to check their 



progress as opposed to tests they conceive for themselves probably related to the ready 
availability of self-conceived tests, the ability to understand the examiner, or possibly 
a spouse and their perception of the degree to which the formal examinations were 
rooted in the real-world and reflected knowledge that students needed to know. 

Most students (71 %) felt that their examinations in the 2 courses 'accurately 
portrayed their knowledge, as opposed to those who were unsure (19%) or felt that 
the examinations did not portray their knowledge accurately (10%). These data argue 
against the evaluation in McKay (Toxicology during the fall semester, 1994: The fifth 
evaluation of the FIPSE-funded innovation in the college of Veterinary Medicine, 
1995), whose conclusions derived from evaluator observations in 30-50% of the fall 
toxicology classes. We will continue to try to elucidate the basis of this difference 
seen by triangulation of the results. 

CURRICULUM REVISIONS: Impromptu ideas for curriculum revision were 

quite perceptive and in some cases, prophetic. These included coordinating surgery 
and medicine courses into systems (5/18; 28%). Interestingly, surgery is beginning 
this revision this fall. The medicine faculty has some support for using this idea 
(50+ %), but is still working on the revision, which is much more complex. Students 
asked for an increase in hands-on experiences for their education (22%). 
Interestingly, an augmented clinical skills course is in its second year. 17% of all 
respondents asked for management and accounting classes. The Hill’s National Center 
for Practice Management is providing these classes (3-4 years) and they are quite 
popular with graduate veterinarians. 

INSTRUCTORS CONDUCTED THE INTERVIEWS: The design of Teaching 
Partners is such that the instructors do conduct the interviews, and their specific goal 
is to improve their teaching. This design has the advantage of being highly personal 
to the students. It has the potential disadvantage of intimidating the students, as do 
all Teaching Partners programs. The underlying theory is that instructors who 
conduct such programs are for the most part highly interested in students, and 
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generally not intimidating. We made a significant effort not to intimidate and would 
have not gone to this work if we were not highly interested in students. The 
evaluators and everyone who has looked at preliminary results of this study have 
raised this question. We agree that it is legitimate for Teaching Partners across the 
country to continue to consider it. 

INCONSISTENCY IN THE NUMBER OF ANSWERS FOR THE QUESTIONS: 

To not interpret beyond the results of this study, we have excluded results from 
questions with only a few answers, or noted both their numbers and percent (7/7; 
100%) when they were included to clarify a trend expressed by other data. In no case 
has a trend been primarily expressed from data of small numbers. Interview questions 
deleted from the set were mostly deleted by students not providing meaningful 
answers and the teaching partners deciding that they should not be included. Early 
in the project these deletions were discussed with Drs Clegg and Shroyer who helped 
us write the questions. Questions added were as the result of provocative answers to 
open-ended questions asked in previous years. We felt that the added questions might 
provide sufficient structure to probe more deeply into this interesting area. When we 
obtain sufficient answers, these data will also be analyzed. 



David, as in the other evaluations, you make several thought provoking points. 
I welcome your response, important to the synthesis of the final evaluation. 
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Introduction 

This analysis was done with a population of veterinary 
medicine students predominantly in their junior and senior year 
of school. There were nine students from the junior class of 

1990- 1991 with six females and three males. The junior class of 

1991- 1992 interviewees consisted of eight females and one male, 
the class of 1992-1993 had two males and three females, and the 
class of 1992-1993 had two males and five females. There was 
only one student (a female) interviewed from the junior class of 
1994-1995. 

Dr. Pickrell, a Toxicology instructor in the College of 
Veterinary Medicine, interviewed the students in a one-on-one 
setting, except for two students who preferred a joint interview. 
There were 11 general questions and 19 specific questions asked 
the participants. Several questions had multiple components. 

The questions focused on areas of: student characteristics, 
factors affecting student learning, examinations, customer 
satisfaction, teacher-courses, and education. It is the 
evaluators' understanding that several questions evolved over the 
course of time and additional questions were added to meet new 
needs. 
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Methodology 

One of the evaluators (Santos) coded the interview responses 
and categorized them by themes. The questions had multiple 
components, some students were not asked all components, and some 
students would not respond to some components. The interviewer 
did not ask the same questions within each year group. Each 
student was asked at least one component of each question. Some 
questions not asked at the start of the study were introduced at 
a later date, and some earlier questions were dropped as the 
study proceeded. 

Results 

General Questions 

l. Why are you going to Veterinary School? 

The interpretation of the question had part of the 
respondents answering as if it were framed in a time context 
(e.g. When did you decide to be a veterinarian?). The other 
portion of the respondents actually answered the question as 
posed. There were 31 students who responded to this question. 

Sixteen responded they knew relatively early in their lives 
they wanted to be a veterinarian. They used reference points 
such as "grade school," "since third grade," and "as long as I 
can remember." Two students knew by high school or early college. 

The remainder of the students focused on the question as 
asked. Seven students mentioned they liked animals and/or the 
profession. Three cited their wanting to be part of the 
community as the reason for their decision. Three were in other 
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fields and transferred to pre-veterinary medicine, and three 
students' answers seemed to vary from the above. For example, 
one student indicated he wanted to combine his interest in 
agriculture and animals, and finally one student wanted to do 
research when completing the degree. 

2 . What is the most important way that your expectations have 
been met? 

Eight of the twenty-nine questioned indicated their 
expectations have been met by the practical experiences provided 
them. Responses included, "I can see it and feel it and then I 
know what is going on." And, "I expected to learn a lot, 
especially the practical or hands-on skills and for the most 
part, our classes do that." 

The next most expressed response was the indication they had 
been challenged. For instance, one student indicated, "It's been 
challenging. When I have a case and can follow it from beginning 
to end and I can diagnose and manage it." 

Two students indicated they felt good because they had been 
able to survive the rigors of school. One student felt the 
image of the school was a positive one. Only one student felt 
his expectations of the school had not been met, but in the 
school ' s defense he recognized this was perhaps because he had 
not yet clearly defined his own expectations for himself. 

2a. What is the most important way we've fallen short? 

The answers here seemed to fall into a pattern depending on 
what year the questions were asked. Overall the students felt 
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they have not been given enough practical experience. At the 
same time, the students felt the practical, hands on experience 
was one of the ways the school had met their needs. (See 
question 2 above) . This theme will be analyzed further in the 
discussion section of this document. 

The second most frequently given responses focused on two 
areas. Four students indicated the following responses as ways 
in which the school had fallen short. They were: areas of 

"professionalism" and professors being "boring." Cited with lack 
of professionalism was behavior of both the professors and their 
fellow students. Examples of this behavior ran the gamut from 
professors talking openly about the administration in class 
lectures, students coming to class late, administrators not 
taking the student's concerns seriously, and fellow students 
being petty about points on an exam. 

What constituted "boring" ran on a continuum from professors 
just standing up and reading to the students to lectures being 
repeated verbatim year after year with little or no change. One 
student commented the instructors were not "top flight." She 
defined a "top flight" instructor as one who "comes to class 
prepared, has good handouts and is enthusiastic about the 
subject. " 

As stated above there emerged a pattern of responses from 
the various classes. Those interviewed in the class of 1990-1991 
predominantly responded that the instructors were boring or 
accented the need for more professionalism. The class of 199 1~ 

O 

ERJC 



50 



6 



1992 wanted more practicum experiences, but no responses 
pertained to the instructors' or classmates' behaviors. For the 
1992-1993 year the respondents primarily cited the lack of enough 
practicum as the way in which the school had fallen short. 

3. Give me some do's enabling you to learn. 

The three areas students tended to focus on were: the need 

for gaining a great deal of practical experience or hands on 
learning, interactive learning, and visual learning. One-third 
of the 32 respondents felt they needed "real world" experience. 
One-third valued interactive learning and five students expressed 
their need for some form of visual medium (e.g. slides, pictures, 
etc.) to facilitate understanding of the concepts presented. For 
example, one student commented, "If you're talking about 
something, do let me see the slide about it. If it's a disease 
process, show me a slide." 

Give me some don'ts enabling you to learn. 

Four students expressed a desire for the professors to "not 
bore them." They indicated they needed not to do "bad lecturing, 
give too much detail, and to not intimidate them." The second 
most frequently mentioned response was students' concerns over 
getting too much detail. One student stated, "Don't give lots of 

minutia " and, to paraphrase another student, two or three 

names for things would be sufficient instead of five or seven. 

In reference to bad lecturing one student commented, “Don t 
use prof anity . . • In lecture I think they could get away from it. 

I can’ t say that I actually learned much from that type of a 
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teaching situation. It turns me off. I don’ t think it’ s just 
me” . 

Another point made by the students was that small group 
interaction needed a facilitator in order to succeed. Some 
students noted they preferred not to have instructors with "bad 
attitudes." Finally, there were individuals who expressed the 
following: not dropping lecture as a modality of teaching, not 

leaving the students without a picture, and not having case-based 
medicine. Specifically concerning the desire to forego case-based 
medicine the student provided an example by saying, "Don't teach 
case-based medicine like... She went from material to case, just 
when you were grasping it, and then back and forth so many times 
that it was really hard to take notes." 

4. Think of a time when you thought, that learning just couldn't 
get any better. Please describe. (Relate this to an especially 
meaningful course.) 

Fifteen of the twenty-nine respondents mentioned some form 
of hands on experience where learning was most meaningful. Again 
in praise of the hands on experience one student stated, "...we 
got to do tracheostomy, thoracotomy tube in recently euthanized 
dogs, open the chest and do cardiac massage. Videotapes 
approached this, but hands on and small groups really does it for 
me. " 

Three felt the use of visuals as helpful in the acquisition 
of knowledge, and two felt it helpful when the teachers are 
motivated and want to be there. Single responses were made of 
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the following: the use of good handouts, learning from mistakes, 

good organization of a course, learning by teaching, and problem- 
based learning. 

4a. Did you feel that way in either of our courses? 

When the students related this question to surgery and 
toxicology courses the majority responded that though they liked 
the problem-based learning technique the more hands on the nature 
of the course is the better they enjoy it. 

One student reflected, "Both classes had a lot of things I 
liked. Toxicology was hard. I read the assignments, but 
sometimes they didn't prepare me for what I ran into. In 
surgery, Dr. Layton gave us this outline, and we didn't have to 
read it right away. It was hard to go back to toxicology case 
(sic) and try and explain it. We could ask anything that we 
wanted and it was a lot of dialogue and that was good." 

Another student expressed his need for hands on experience 
over the problem-based learning like this, "No, remember, I need 
live animals... your cases are closer than didactic lecture and 
videotapes closer yet, but not the real cigar." 

5. What is your best learning style? 

One-third of the responses were consistent with the pattern 
of practical, experiential learning identified above. The 
students felt their best learning style included some form of 
hands on training. Six of the twenty-eight respondents stated 
some form of group setting helped them learn well as indicated by 
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the following comment, "I learn best be studying in a group. For 
me it has to be a small group." 

Problem-based learning was specified by four of the students 
as their most optimal way to learn. One student in the class of 
92-93 related his experience with problem-based learning by the 
following comments, "I learn more from problem-based 
learning. . .This year's rotation was preferable to last year’ s 
clinical toxicology. The instructor can guide you." 

Three other types of learning were named each by two 
students. They were: visuals, good lecture, and good 

organization. One student indicated the need for the course to 
present a challenging environment, and still another felt 
interaction with the individual instructors to be the best way to 
learn material. 

6. Someone has recently said to me, that learning difficult 
material, or solving difficult problems is hard. How do you 
attempt to answer these problems? 

The students’ responses were multiple and varied. The use 
of rule out lists was included in five of the twenty-five 
respondents' answers. Comments that focused on the eclectic 
approach include, "I use consults frequently and work through a 
rule out list." And, "I do use laboratory data, consults, rule 
out list and even mind set." 

The use of consults (instructors or fellow students) was 
mentioned by five students as well. Generally responses of the 
students tended to signify their intense scrutiny of the problem. 
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The problem solving process seemed to have them use multiple 
resources to get the job done. One student’ s answer held multiple 
responses. “Chip away at the problem a piece at a time. 

Redefine. Keep looking. I learn by reading. Then I’ d call 
somebody. I like to solve the problem myself, then I can use it 
again" . 

6a. Would it help you as students if at the start of toxicology 
we talked a little about how you were going to run into prelims 
which were going to be difficult, but not impossible to solve, 
and I thought it might be important to tell you how to go about 
solving them. 

All but one of the students thought this would be a positive 
addition to the toxicology course. Only one student felt that 
explanation wouldn't be helpful but did not elaborate. 

6b. Unrelenting instructor support facilitates instructor- 
student interactions which shape your capabilities. Do you 
agree? Can you provide an example? 

All of the seven questioned here agreed with the comment. 
Their reasons varied but predominantly focused on the support 
this provided them. One student stated, "Agree. .. lots of examples 
made me feel more confident ... important because then I felt that 
people cared." Another commented about effects on confidence "if 
the instructor is not excited or motivated. . .then it's hard for 
you to get excited... in radiology teachers are open minded, 
listen to questions, seem to enjoy working with students even if 
repetitive questions are asked." 
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7. What do you view as the purpose of an examination? 

The responses varied but most focused on the exam being a 
time for students and the instructor to know if they were "where 
they were supposed to be," or to assess what they had learned. 

One student even insinuated it was a learning process for the 
instructors as well. "I think it is to see if the student is 
grasping the material (and whether you are getting your ideas 
across). Will they go away with the material, being capable or 
competent of using that knowledge in practice or surgery... It 
also does something for you, because we reflect back on you and 
what you taught us (you are known by the quality of the students 
that you produce) . " Another student coupled his serious response 
with a bit of humor. He stated the exams were, "To check that 
each student has a minimum data base to go to the next level... or 
maybe to give us ulcers." 

8. Are students and instructors partners, antagonists or 
disinterested parties in examinations (Can we see each other from 
the other’ s point of view)? 

Fourteen of the 25 respondents felt the relationship was one 
of partners. "I think they are partners. I think that’ s the way 
it should be. The instructors are here to teach, the students 
here to learn. That binds them together right there" was one 
student’ s comment that effectively summed up the others’ 
responses. 

The second most frequent response (six of the 25) was that 
there was a mix of how the students perceived this relationship. 
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In other words they felt they were partners sometimes, at times 
antagonists, and at times disinterested parties. Most of the six 
indicated it was instructor-dependent, citing their claim with 
phrases like, ".. .depending on the instructor," or, "...I think it 
depends on the instructor." Two respondents felt the 
relationship was strictly an antagonistic one, and still another 
student indicated that the relationship was primarily one of 
disinterested parties. Little elaboration existed to explain the 
last two categories of responses, but the students indicated the 
antagonism or disinterest was dependent on the instructor. 

9. How do you study for an examination (groups, individual, 
keep-up or cram {24 hour recall, 48 hour erase})? 

Almost one-third (nine of 30) of the respondents indicated 
group work was their predominate way to study followed by eight 
responses indicating they studied by themselves. As for the cram 
or keep up issue there was a mixed response. Six felt they 
crammed, and three felt they did both. 

A pattern emerged when you looked at the responses sorted 
according to "year group." In year group 90-91, eight of the 
twelve students indicated group work was their preference. None 
of the eight students in the junior class of 91-92 cited group 
study as their preference. Their responses mainly ignored the way 
they studied best (e.g. individual or group) and focused on the 
cram or keep up issue. Their answers were mixed with three 
stating they used both the cram and keep up methods and two who 
primarily crammed. Three felt they studied best as individuals 
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and only one of these students reported keeping up all along. 
Again, none of this class stated a preference to study as a 
group, but one individual did report studying in a group after 
studying alone around 24 hours prior to exam time. 

Out of the nine students in year group 92-93 four indicated 
a preference for individual study. Only one chose group study, 
and one indicated liking to study with a small group after 
studying alone. Three felt they more or less crammed for the 
exam, and two felt they tried to keep up. 

The one student in year group 94-95 indicated her primary 
modality is to study alone. She did indicate she studied with a 
specific person sometimes prior to examinations. She did not 
respond to the issue of cramming or keeping up. 

10. How do you evaluate (check) your own progress (daily# 
co ur se# year# veterinary medicine)? How will you corroborate 
your estimates of that progress? 

Half of the respondents (n = 14) felt they administered 
various forms of "self tests" such as the ones offered here from 
the interviews. One student said, "The best way for me to 
evaluate my progress is when somebody is speaking to me or 
they’ re lecturing, if I know something about what they are 
talking about, then I figure I have learned something." Another 
student commented, "I test myself. I read my journals and see if 
I really understand the material." 

Five students used the official examinations to gauge their 
progress . One student commented this was indeed how he checked 
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himself but offered ambivalence at this approach. "The worst way 
is to check your grades when you get them back on the test, and 
that’s how I do it. This is frustrating." 

Four indicated they did not know how they were checking 
themselves. Comments that reflected this tone included, I don t 
think that I have solved how to do this" and another said, “I 
haven’t a clue. I really try, but don’t get too far." Still 
another responded simply, “I don’t know. 

ll. is the teaching partners program in which Dr. Layton and I 
work together to improve teaching with your help meeting your 
needs for education? How specifically has it helped you learn? 
How do you feel about this cooperation? 

A resounding 24 of 27 respondents felt the program met their 
needs. Two did not answer the question as posed. Mostly the 
students said these instructors were not just "putting in their 
time." They saw this program as very positive in their 
education, comments included, "All the instructors need to be 
doing it," and "Wonderful!! You keep us (and yourselves) excited, 
focused and moving toward a goal." 

Specific Questions 

1. In what important way have your expectations for Clinical 
Toxicology or Surgery been met? What’ s the most important area 
for potential improvement? 

Ten of the 26 students felt their needs had been met in 
these courses. They indicated in one form or another that they 
had learned a great deal. Three students felt the hands on 
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approach (especially in surgery) had been helpful, and three 
indicated the practicality of the classes had aided them in 
building on their knowledge base. Two students felt this type of 
class had given them a needed “confidence” and one indicated more 
confidence to problem solve. All classes responded in a variety 
of ways, and no pattern emerged within the distinctive classes, 
la. What is the most important area for potential improvement? 

Seven students felt there was simply too much material with 
no clear indication of what was crucial and what was "nice to 
know." Several students indicated a clear frustration with the 
amount of material to be covered and tested on. Statements like 
the following indicate frustration over the workload: 

“It is important for us to be exposed, but maybe it would 
have been more helpful to emphasize the major toxins ...that I 
will have to handle versus a question over a more minor toxin 
that I just happen to know..." 

“Surgery: disappointed in the content, material overload..." 

Five students felt there was a need for more surgeries to be 
included in the curriculum. Again, this view seemed to mirror 
the above statements about the sheer amount of material to be 
covered in one semester. One student said, "I felt that surgery 
last semester was a blur. I picked up on her dissatisfaction and 
I was dissatisfied as well." 

Three students felt there was a need for more minilectures 
in toxicology. All three of these came from the class of 90-91. 
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There were four groups of two students each that offered the 
following suggestions in regards to improvements: a more 

thorough syllabus, more continuity with teaching style throughout 
the courses, too much material in toxicology, and less subjective 
grading in toxicology. One student commented everything was OK 
and there was no need for improvement. 

2. What did the teacher do that helped you to learn? 

Twelve of the twenty-nine students who responded indicated 
some form of interactive learning aided them in class. The more 
interactive the better, and the more interactive induced the 
students to view the instructor as involved and caring about 
them. They used words like "feedback," "interaction," and 
"questions and answers." 

Four students felt the use of good outlines and written 
cases helped them learn while another four liked the use of 
minilectures. Three enjoyed the atmosphere where the instructors 
were "not talking down to them." They felt it aided their 
learning. Still another three thought the use of the quizzes 
helped them to learn. A final two students indicated some form 
of visual aid was helpful. One response was indiscernible. 

3. Do you think it is important that the majority of surgical 
diseases or specific toxicities are actively discussed (as 
opposed to having assigned reading) in class? If so, how do you 
envision teaching all of these topics in the allotted time? 

Almost half or 13 of the twenty-eight students who responded 
felt the need was there to just teach the common ones or the most 
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important ones and if time allowed the others could be explored. 
Some students (five) indicated it would be nice if the important 
cases would be placed in the library and made available to them. 
Three indicated it would be nice to use toxicology as a model and 
try and discuss them all. Two felt the use of good notes 
supplied by the instructor would help in this endeavor, and one 
student envisioned learning the copious material by hands on 
experiences. 

4. Good surgery is considered to be an art. What fraction of 
the course teaching you that art should be devoted to the 
veterinary medical decision making process (diagnosis ) , as 
opposed to an exact description of the surgery to be performed 
(technique)? Why? 

Twenty of the 27 students felt there should be placed a 
larger amount of time on diagnosing, especially the common ones. 
Three students indicated there should be more time spent on the 
techniques of surgery. Two students indicated there should be 
equal time in toxicology on the common and uncommon ones and 
equal time in surgery devoted to technique and diagnoses. Two 
students’ responses feel out as "non answers”, and one indicated 
sophomores should do some basic steps in surgery to get oriented 
to the process. 

5. Some people feel that it is the responsibility of students to 
learn and instructors to teach. How do you react to this 
feeling? 
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Sixteen of the 26 respondents felt both the teachers and the 
students should do both. One comment summed up their sentiments, 
"Don't box us. Instructors and students can do both." Four 
students agreed with the statement as presented in the question. 
In other words they felt it was an accurate portrayal of the 
relationship. There were three responses that seemed to not fit 
the question, and one student indicated students should learn but 
gave no thought as to the role of the teacher. The responses 
were spread evenly across the groups. 

6. What suggestions do you have to make it even easier to learn/ 
or for you to learn even more of what will be practical to you in 
your practice? 

Almost one-third or seven of the twenty-two students 
responding indicated it would be beneficial for them to have more 
hands on or practical experience. They prefer learning by doing. 
For example, “More clinical skills or hands on... as juniors we 
should be in clinics half days... then we won’t stumble over the 
paperwork" . 

Six students felt there was a need for more case studies and 
problem solving with the cases. This thought ran in all year 
groups. Two students each replied the following would help them 
in their practice. They were: more minilectures, more tracking, 

and providing challenges to them. One student stated there 
needed to be an effort to better organize the material and 
provide a better syllabus. One indicated the teachers should 
not put them down, and one had no idea. 




7. What did you like best about the teacher or course? 

Nine of the seventeen students responding indicated in one 
form or another they felt the instructors treated them with 
respect and/or really cared about them as individuals and 
students. Comments like, “Some teachers are interested in what 
they teach, you in what we learn" seemed to be echoed throughout 
their answers. 

Four students felt the minilectures were advantageous. Two 
liked the oral exams, and one again indicated enjoyment of the 
hands on approach to learning. 

7a. What did Dr. Pickrell do to improve your performance on the 
oral exam? Did it help you learn more toxicology, or were the 
questions just easier? Did the graded difficulty help you to 
learn more, or perform better? 

Only six students responded to the question. Three 
indicated he helped them get started. For example, one student 
said, “Getting started on the oral examination is the hardest 
part. . .making me feel like I'm on the right track helps..." . 

One student felt that Dr. Pickrell did not help his performance, 
one indicated he helped by not being intimidating, and one had no 
opinion. 

7b. How important is instructor support to you as a discussant 
in clinical toxicology? How can we make you feel more supported 
and allow you to make your mistakes “with a net"? 

All four of the respondents felt it was very important for 
the teacher to support them. Only two gave suggestions as to how 
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best this can be achieved. They were: to not be intimidating, 

and support them even when correcting them. 

7c. Everyone makes mistakes. How would you like to be rescued 
when you make your next one so that everyone knows the best 
answer? 

All seven of the students felt in some way they would want 
to be rescued by someone who was not going to yell or intimidate 
them. One added the teacher could also do this in a polite 
manner. 

7 d. How important to you was following the seven features of a 
clinical cases? 

The seven features of a clinical case are: identification 

of the problem, listing of possible diagnoses, clinical testing 
to be performed (e.g. lab, x-ray, etc.), diagnosis or 
confirmation, initial treatment of the problem, prognosis, and 
client education. Others features may be important such as 
environment and sources, but the seven above are crucial. 

All seven students responded they felt the seven features 
are important. One added that the addition of rule out lists 
would be helpful, and one indicated the seven features helped 
give her structure. 

8. What did you think was an area for potential improvement? 

The responses were varied and more evenly distributed than 
previous answers. Students seemed to have diverse views about 
improvement. Specifically four of the sixteen students 
responding felt in some way the tests were too difficult and 
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needed to either have fewer cases on them or more time to take 
them. Three students liked the idea of having more cases 
presented to them for clearer understanding of the concept being 
presented. Two students in each of the following indicated their 
ideas for improvement. They were: more hands on, staying on 

track or keeping focused, and giving the student a mental or 
physical picture to work with. One felt improvement could come 
with teachers not reading slides, one liked Drs. Pickrell and 
Layton's approach to teaching, and one wanted the teachers to 
push them more. 

9 . What is your perspective about this course or teacher? 

Twelve students answered this question, with 10 indicating 
satisfaction with both of the courses. No delineation was made 
by the students in regards to their liking one of these courses 
over the other. Grouping of the two courses (surgery and 
toxicology) seemed the norm. Comments such as, "Overall very 
good, I enjoyed both," indicated the students saw the style of 
these courses as similar and felt free to respond about them as 
one. Two students indicated they felt they had answered this 
question with their answers to other questions. 

10. How do you think or feel about the course-teacher? 

Eight of the students again indicated their positive 

feelings about these courses and again no delineation of the 
courses as separate entities. One answer was interpreted as a 
negative response because the student indicated feeling put "on 
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the spot." And one simply repeated the question and didn't 
answer it. 

10a. Did you have a lot of trouble giving up your opinion once 
you had formed it? 

Twenty- four students answered with greater than half of them 
feeling they were able to give up their opinion once it was 
formed. Many qualified this response by indicating in some way 
they needed to be persuaded by valid information and facts before 
they would do so. Six students felt they could sometimes change 
their mind depending on the situation, and three stated they 
found it difficult to change their mind once they had decided 
about something. 

11. What are the benefits (of these courses) to you? 

Four of the nine students felt the benefits included the 
ability to problem solve and organize themselves. Four also felt 
they had learned a great deal from these courses. One simply 
stated the style of going over the cases instead of lecturing was 
more desirable for them. 

12. How can we make it a learning experience for the faculty as 
well as you? 

Of the fourteen students who answered the question, nine of 
them felt the teachers could learn as they taught. Three did not 
answer the question, and one believed if the faculty had access 
to surveys such as this they could benefit from it. 

13. In this course/ how often do you read the texts? 
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This was divided into the toxicology and surgery texts. For 
toxicology eighteen of the twenty-six students indicated they 
read the text, but there was no clear indication of the 
readability of this text. For the surgery text six students 
indicated they read it, one did not, and eight read her notes 
more. Their comments were most flattering about her notes. Most 
preferred them over the text. 

14. Did the examinations in this course accurately reflect what 
you had learned? 

Fifteen of the 21 students indicated they felt the exams 
reflected their knowledge. Four were unsure, and two did not 
feel it was an accurate portrayal of their knowledge. 

15. xf you were in charge of the universe for fifteen minutes 
and asked to design the ideal examination situation, how would 
you change from what was done in this course? 

It was not easy to tell exactly how the students felt about 
this issue. Eleven of the 35 respondents felt the oral exams 
were positive, and four indicated that within this context the 
case histories were good. Summing up the views of the merit of 
oral examinations a student commented, "More orals would be 
especially helpful, they helped me to learn the most of any 
examination. Written examinations are a poor substitute, but 
essay exams were good too." One student felt the use of 
performance exams were "the way to go." 

As for the design and frequency of exams, few students 
responded to this portion of the question. Two students felt the 
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number of tests were too few due to the volume that was covered 
at each exam, and one student didn't like multiple choice citing 
they were, "deceptively simple." 

15a. Curriculum Revision? 

Five felt there needed to be more organization within the 
courses, but one disagreed. "Grouping systems together (or 
coordinating lecture-discussions so that all of one system is 
covered) during one period is probably not a good way to go. 

After six weeks of lung or heart, I'd probably be sick of hearing 
about it and not pay too much attention." 

Four felt there could be an increase of hands on experiences 
provided them, and most did not elaborate on how this could 
occur. Two students felt that oral exams should be increased 
across the curriculum and three felt a need for management and 
accounting classes would be advantageous since most will be 
owning a small business themselves through their practice. One 
felt there was a need for more surgery time and two others 
desired less neuroscience within the curriculum. Finally one 
student indicated he thought it a good idea to let the students 
stand up after 20 minutes of lecture like one professor did. 

16. A prominent educator has said Education is what remains 
after you've forgotten the specific details. What learning will 
“stick to your ribs" from clinical toxicology or surgery? 

All of the fifteen students either overtly agreed with the 
assertion or within the essence of their answers indicated they 
considered the statement to be true. All felt both of these 
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courses had given them something to “stick to their ribs" . One 
student gave an example for each course of what she had 
remembered. She said, "In toxicology, we got the terminology and 
where to look it up. Some common toxicants. I'll always 
remember nitrate is brown and cyanide is red. Just the specific 
procedures that Dr. Layton has taught us that we might be able to 
do almost in our sleep." 

17. What do one of us do that the other one might benefit from? 
Where, indeed, should one put the minilecture? 

There was very mixed response concerning the question, and 
some students did not answer it as posed. Ten of the twenty- 
eight students felt the minilectures were important and the oral 
exams were positive. It was difficult to tell who the students 
were talking about with some of their responses. Specific input 
from one student indicated that Dr. Layton had the students react 
faster and this could help Dr. Pickrell. They also thought the 
use of Dr. Layton’ s hand outs would benefit Dr. Pickrell. For 
Dr . Layton one comment was she needed a better room and if she 
had more time she could do more group work. 

17a. The organization issue? 

Twelve of the twenty-three students gave their impression as 
to how and what would constitute better organization for them or 
the course. The responses varied from personal approaches to 
more global ones. They were: summary sheets, coincide surgery 

with medicine and maybe even toxicology, do all of one system and 
move on (e.g. GI, etc.), charts, take a little at a time, ordered 
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the reading, [that is, the student gives different priorities to 
what she has to read], organize by signs, help us tie it 
together, read notes and cases, examples would help, we have to 
organize ourselves. Seven students had no problem with the 
organization in either course. 

18. When extending common diagnoses to the more complex or 
difficult ones, what similarities can we build on, and what 
unique differences do we need to learn how to handle to make 
competent medical decisions? 

The answers were as varied as the students themselves. 

Three did indicate the use of minilectures would help them in the 
future. Some students seemed not to answer the question. The 
following answers are some suggestions from the students 
themselves. 

“There is no substitution for clinical experience." 

“If you know the common stuff very well, you will have a 
model of the common stuff and you will see it not fit." 

“ I pick out the big differences to help me to 
differentiate." 

“First you need to recognize there is a problem. You need 
to list the more common differentials, don’ t chase the 
zebras, keep them in mind but focus on the common." 

19. Senior multimentoring, and to a certain extent clinical 
toxicology and surgery may be viewed a beginning to engage in a 
two way conversation between student and teacher • How does this 
engaging in this process make you feel? 
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Seventeen of the 35 respondents indicated in some way this 
was a positive experience. Five signaled some apprehension with 
this at first but with time this eased. 

Discussion 

The general questions one through 11 seemed to focus on 
information that students in this type program could answer with 
some degree of expertise. They were questions that explored 
their background before school and elicited information about 
their preferred learning style and examination preferences. 

The second section of questions were more specific to the 
classes in question (e.g. Surgery and Toxicology) and the program 
in general. They were thought provoking and somewhat 
philosophical in nature. One might question the merit of having 
students at this level critique a program when they have little 
on which to base a comparison. 

As a case in point, consider the constant referrals to more 
hands on work desired by the students. Many students in fields 
like veterinary medicine want very much to "get on with it" and 
perform tasks associated with the profession. It takes the 
professors' diligence and patience to reinforce that along with 
the practical work involved also comes the theory. 

The students recognized a positive part of the program 
(question 2, general section) as the hands on experiences given 
them and in the very next question stated the way in which the 
program had fallen short was with the lack of hands on 
experiences. Findings such as these indicate the students’ 
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knowledge deficit of what goes into the making of a professional 
medical practitioner. If they were to design the program one 
would have to be concerned their design would be no more than a 
vocational/ technical school environment steeped with much 
practicum and little theoretical base. 

Another concern was the fact that one of the instructors 
conducted the interviews. This perhaps was not a problem for 
most of the students, but some might have considered the 
consequences of not responding positively about the courses or 
the program. The answers appeared to have been candid, but we 
will never know. 

Finally there was the inconsistency of the number of answers 
for the questions posed. Not all interviewees responded to all 
questions. It would have been beneficial to pose all questions 
to all participants. The evaluators recognize the possibility 
that students refused to answer certain questions posed. 

Discussion with Dr. Pickrell highlighted how valuable he 
found the information obtained in these interviews. He described 
the data as "rich." It is not clear to what extent these data 
proved useful in terms of curriculum decisions. 

Some interview questions were added as the interviewing 
progressed. Some questions asked in initial interviews were 
later dropped. A varying data base proves more difficult to 
interpret, and the variation in questions asked of separate year 
groups presents incomplete responses — even missing data one 
could argue — in several instances. At the same time, the 
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evaluators recognize that issues emerge as a study develops, and 
not to respond to emergent needs is unprofessional. We do want 
to accent, however, that placing significance on some responses 
that have fewer or even few respondents does inhibit full 
disclosure of the themes presented and even may become misleading 
as undue emphasis is placed on an anomalous response. 
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